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ABSTRACT 

The Army, in a period of constrained resources and 
increasing demands on its leaders, can ill afford to pursue 
leadership training which is ineffective. The evaluation plan 
developed in this study seeks to provide the decision maker with 
information necessary to guide the training development 
towards its desireed outcome: producing better leaders. 

A review of the leadership theories contributing to the 
Army's organizational leadership model, their training programs, 
and the leadership training of the other services is presented. 
Their methods of program evaluation are studied. 

The evaluation plan is a systematic study employing five 
principal criteria; process evaluation, learning, attitudinal 
change, behavioral change, and the change in organizational 
performance. Each is discussed in terms of its source of data, 
experimental design, and contribution to the overall under- 
standing of the training program's effectiveness. The evaluation 
scheme is presented in an action plan format to coincide with 
other ongoing initiatives in the leadership and educational fields. 
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I. THE NEED FOR EVALUATION 



A. INTRODUCTION 

The military forces of this nation, as a microcosm of 
society, can look to a future of rapid change along all 
fronts . In order to develop plans and programs which will 
be carried into the new century these environmental trends 
must be plotted and dealt with now. Within the next twenty 
years the 13 to 24 year old population will decline by 217,. 
This will require that almost 507, of all male high school 
graduates be recruited by 1995 to meet authorized strength 
levels as compared to 407, in 1980 [US Army Strategic Studies 
Institute, 1978, p. 623. The soldier of the 80 ' s and 90's 
will serve in a technology and system intensive organization 
whose objective will turn from equipping the man to manning 
the equipment. Demands for technicians will outnumber the 
supply as new equipment is placed into the field. Into this 
environment he/ she will bring an increased demand for indi- 
vidual rights and independence, and will be more vocal about 
perceived abuses of his/her time and talents. 

Officers will receive more specialized assigrmients , be 
rotated less frequently, and advance without having held 
command leadershio oositions. 



Leadership and mission accomplishment will be functions 
of technical expertise, qualification in multiple skill 
areas, and cognitive ability to analyze, sort, and se- 
lect alternative courses of action from wide varieties 
of digital output. At the same time, leaders and per- 
sonnel managers will not be able to exert control or 
influence activities simply through the use of directive 
authority. A leader's success in motivating and directing 
subordinates will be based on the ability to use a varying 
combination of leadership skills and approaches effec- 
tively in a variety of management situations. 

IjJS Army Strategic Studies 
Institute, 1978, p. 6^ 

To prepare our leaders to meet the challenge is the task 
of the various military training systems. This training will 
be plagued with budgetary restrictions as funds go into more 
sophisticated hardware. The result will be fewer courses 
offering long term resident instruction, greater reliance on 
exportable training packages, and short-term (temporary duty) 
courses stressing technical skills. This course of action 
began several years ago in the Army following a study of the 
1973 Mid-East war. 

The U.S.Army Training and Doctrine Command began revising 
school curricula to emphasize technical competence in the 
operation and employment of our weapons systems. Funds, 
however, were not available to lengthen the courses. The 
result — some of the more traditional subjects relating 
to officer development were dropped out. 

(Review of Education and 
Training for Officers, 1978, 
p. v] 

This philosophy towards training will fail to prepare 
the leader (commander, leader, manager) to deal with the 
greatest resource available, people. Leadership training 
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in the Army has not been 'dropped out' or cut back due to 
a lack of importance, but because a baseline for leadership 
doctrine development activities has not been formally estab- 
lished and a failure to establish a correlation between what 
was being taught and the resulting tangible benefits in the 
operational units . 

Leadership, more than any one factor, must serve as the 
cornerstone for shaping the Army environment of the 1980' s. 
It will be leadership which provides us with the vision 
and inspiration to deal with some of the most complex 
problems to confront a peacetime Army in our country's 
history and simultaneously prepare our soldiers for war. 
Leadership makes things happen. It is leadership which 
creates and shapes unit environments which inspire soldier 
motivation and commitment; it is leadership which shapes 
unit cohesion; and it is leadership which must work col- 
lectively to shield soldiers and units from the dysfunc- 
tional forces and tendencies which are natural spinoffs 
of centralization and technical innovation. Thus, we 
must consciously direct our attention and commitment to 
leadership training and development if it is to be the 
force which creates an environment conducive to training 
and molds our soldiers into units prepared to go to war. 

[Rojas, 1980, p. 33 

As the 1980 's begin there exists an opportunity to 
rectify this problem. A doctrine for leader development 
has been promulgated and a Department of the Army level 
conference convened to plan for its implementation. The 
subject of program evaluation remains to be addressed in 
order to complete the transition to a more effective Army. 



B, THE NEED FOR EVALUATING LEADERSHIP TRAINING 

All programs eventually reach a point in their implementa 
tion where the designer or manager evaluates its relative 
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effectiveness against some established criterion. Just as 

the project manager would be remiss in failing to analyze a 

project not yielding its expected rate of return, why then 

doesn't this realization of importance of evaluation extend 

to training? McGehee and Thayer [19613 drew an analogy 

between training evaluation and Mark Twain's comment about 

the weather, i.e. "everybody talks about it, but nobody does 

anything about it." [Blumenfeld and Crane, 1973, p. 42] 

A quick review of the training literature however, reveals 
the practice of evaluation has not kept pace with the 
prescription. Evaluation research is conspicuous by its 
absence and when present is frequently remiss in applying 
appropriate controls to draw reasonably valid conclusions. 

l^unker and Cohen, 1978, p. 4] 

Organizations, reluctant to assess training programs, 
frequently cite one or more of the following reasons for 
no t e va lua t ing : 

1. Lack of funds, evaluation is too expensive. 

2. An unshakeable -belief that their training works. 

3. It is impossible to get accurate results, so that 

What is really being said is: "We are afraid that the eval- 

uation will show we are doing something wrong or have made 
the wrong decisions and we would rather not know about it." 

The hazard of this 'head in the sand' attitude is similar 
to a type I or type II error in statistics: type I, accepting 

the program when it is bad and suffering the consequence of 
continuing to spend considerable sums of money and time on a 
program which is not contributing to the effectiveness of the 
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organization; or, type II, rejecting the training program 
when it in fact is benefiting the attendees and the organ- 
ization. 

When an organization does realize the potential benefits 
of a comprehensive evaluation plan and includes it as a 
significant phase in the training system, it can profit in 
many ways. Some of the reasons for evaluating training 
include: 

1. Assess achievement of training objectives. 

2. Assess effectiveness of the trainer. 

3. Justify the expense of training through cost-benefit 
analyses . 

4. Improve the content/ structure of the training. 

5. Decide whether other trainees should receive the 
training . 

6. Identify which trainees benefited the most/ least. 

7. Reinforce major points for the trainees. 

8. Create advance expectations in the minds of the 

trainees through utilization of pretraining tests. 

Evaluation answers the question what is it worth, not how 
it works. The purpose of evaluation should be to support 
decisions to initiate, modify, maintain, or terminate 
various courses of action. 

[Eoyang, 1977, p. 3] 

The fundamental application of evaluative techniques is 
twofold: to aid the decision maker by clarifying the some- 

times ambiguous results of a training program, and secondly. 
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to provide feedback to the individual on how well he/ she has 
succeeded in meeting the objectives of the instruction. 

General Douglas MacArthur wrote, "In no other profession 
are the penalties for employing untrained personnel so 
appalling and so irrevocable as in the military." [Pitts, 

1976, p. 19], This is particularly true in the area of leader- 
ship training. In determining the need therefore for the 
evaluation of leadership training the question must not be: 
"Can we afford to assess our training efforts?" but rather; 
"Can we afford not to?" 

Assuming the importance of conducting an evaluation the 
next issue becomes one of where the assessment fits into the 
overall training scheme. Obviously the evaluation plan must 
be supportive of the objectives promulgated in the training 
program. To assist in this step a systems approach should 
be taken to integrate the evaluation. It must be realized 
that not only is evaluation a subsystem of training but that 
training is a subsystem of the organization and the environ- 
ment in which it functions . As such any evaluation design 
must reflect the interrelationships of the process with other 
components of the system, e.g. the personnel management 
system, service school system, and even the customs and 
traditions of the organization as a system. Failure to con- 
sider the ramifications of this will lead to a disjointed 
scheme of evaluation, lack of commitment to the plan, and 
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the eventual disregard of the findings of the evaluation. 
System integration is the key to evaluation design. 

Isolating for the moment the training process, evaluation 
is typically the final step. The US Army, for example, has 
identified seven major processes within the training system; 

1. Policy 

2. Long-range research 

3. Concept development 

4. Training development 

5 . Training 

6 . Operational management 

7 . Evaluation 

In order to support the first six stages evaluation must 
focus on four factors in order to provide meaningful feedback 
to the decision maker. Alden £19783 describes these factors 
as ; 

1. Willingness to Change* 

If no decision concerning a change in policy, 
research, etc. will be made as a result of the evaluation 
then the design and implementation of an assessment plan is 
a moot point. The Army recognizes the importance of a 
thorough evaluation as evidenced by the inclusion of this 
process in the Army Leader Development Plan. The plan, an 
output of the 1980 Action Planning Conference, is currently 
in the draft stage. Commitment to evaluation remains 
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contingent upon the viability of the assessment design. 

2. The Type of Information Needed* 

The type of information can best be determined by 
returning to the importance of systems to training and eval- 
uation. Viewing it as a system comprised of inputs, process, 
and outputs, the research questions center around the relation- 
ship of these variables. For example, if the decision 
involves the presentation of material then an examination of 
the process is appropriate. If the decision maker needs to 
know whether or not the program should be continued then a 
complex evaluation answering questions about the change in 
both trainee attitude and behavior and change in the organ- 
ization in which the trainee functions (or leads) is called 
for. The time frame available to the decision maker in which 
to reach a conclusion is critical to this question. 

3. The Source of the Information. 

Campbell, Dunnette, Lawler, and Weick [1970^ discuss 
two sources of data, internal and external. Kirkpatrick^ 

£19793 offers four sources; reaction, learning, behavior, and 
results. Implied by these levels is a requirement to address 
such specifics as experimental design and constraints on the 
data gathering. The relationship between these two thoughts 
and their implications for evaluation comprises a major 
portion of this thesis and are discussed in detail in 
Chapter IV. 
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4. Decision Criteria. 

\7hat information will signal the decision maker that 
a change should be made? This can be ^<rritten in terms of some 
absolute standard such as a paper and pencil test on material 
taught, or a statistical comparison between control and exper- 
imental groups. \^ichever standard is used, the decision 
maker should be prepared with some course of action before 
the results of an evaluation are kno^^m. 

Recognizing the importance of conducting an evaluation 
and the focus or direction such a plan must take, it remains 
to be emphasized that the development of an assessment plan 
needs to coincide with the preparation of the training pro- 
gram itself. This requires that the fear of the evaluation 
yielding negative results be overcome by the trainer and the 
decision maker. This thesis is directed towards preparing a 
viable evaluation model at a time when the Army is embarking 
on a new course in its leader development. 

C. THESIS VALUES 

The U.S. Army, like all organizations, is a values based 
institution. Values lay the foundation upon which goals and 
behaviors are specified and the future state of the organ- 
ization is built. As James MacGregor Bums states, "Mobilized 
and shaped by gifted leadership, sharpened and strengthened 
be conflict, values can be the source of vital change." [1978, 
p. 4l] Accordingly, in order to reach its future state, this 
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evaluation plan is built around the following values: 

1 . Goal-Directed 

The evaluation plan will clearly specify tasks 
necessary for implementation, detail the rationale behind the 
tasks, and provide a meaningful product to the Army's leaders 
and training developers . 

2. Consistent 

The plan will take a systematic approach. The train- 
ing development system, Army school system, the personnel 
management system, and a recognized system of evaluation must 
be systematically associated. There must also be consist- 
ency with, and reflection of, the objectives and goals of the 
leadership training program. As mentioned previously, system 
integration is the key to any evaluation, it can not be 
written in isolation. 

3 . Innovative and Creative 

The plan should not be constrained by the phobias 
which have frequently affected many evaluation attempts in 
both the military and civilian training programs, nor will 
it be constrained by the argximent "That isn't done in the 
military, no one will buy it." It will question the value 
of tradition where it is seen as an abstacle to progress. 

4. Pragmatic 

While many programs can be creative, their "pie in 
the sky" approach can doom them to oblivion. A program must 
be practical as opposed to idealistic. The decision maker 
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must be presented with a step by step outline to guide him 
or her through the evaluative process. This value is closely 
related to that of Goal-Directed. 

5 . Flexible 

Recognizing the climate in which the military operates 
as being moderately dynamic, subject to budgetary uncertainties, 
the plan must provide for contingencies. 

The principle benefit in establishing the above values 
and norms is that, if a proposal seems unclear or difficult 
to specify, its appropriateness for inclusion can be judged 
by the stated objectives and values. 

The objective of this thesis is to develop a viable 
evaluation plan for Army leadership training which will 
provide feedback to training developers and decision makers 
on the effectiveness of the program and to leaders on their 
style of leadership. 

The plan is not devised to evaluate leaders, that 
remains the job of the superior. Nor will the thesis 
question the appropriateness of the current leadership train- 
ing programs. The purpose of the plan is to provide a 
strategy by which the program's effectiveness can be reviewed. 

To reach this objective the following organization will 
be used in developing the plan. 
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D. DESIGN AND ORGANIZATION OF THE THESIS 

This introductory chapter has focused the reader's atten- 
tion on the environment within which the military must operate 
throughout the remainder of this century, its particular 
implications on leader development, and the need to monitor 
those training programs designed to reach effective leader- 
ship . 

Chapter II sets the stage for exploring the current state 
of leadership research through a presentation of those theo- 
ries of leadership which have had the greatest impact on the 
military's approach to leadership training. Those theories 
which offer a training program will be examined particularly 
closely to ascertain what method of evaluation, if any, was 
used. 

The 'art' of leadership in the armed forces, being rather 
unique from that in the civilian sector, requires that a 
unique approach be used in the development of its leaders. 
Accordingly, the approach taken by each service is reviewed 
in Chaptejr,JII-j — Particular emphasis is placed on the US Army 
and those systems with which the evaluation plan must inter- 
face. 

Kirkpatrick's criteria for the evaluation of training 
will serve as a base for the various methods of assessment 
available to determine a program's effectiveness. The advan- 
tages, disadvantages, and utilization of these techniques 
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are presented in Chapte r IV. 

Chapter V organizes the material presented in the fourth 
chapter into an action plan format. This standardized form 
clearly specifies the action needed to be taken, the agency 
responsible for the action, and, in accordance with the Army 
Leader Development Plan, establishes milestones for each 
action. 
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II. REVIEW OF LEADERSHIP LITERATURE 



A. INTRODUCTION 

The subject of evaluating leadership training must, by 
necessity, commence with an overview of the theories of 
leadership. Once the subject is placed in perspective an 
examination of contemporary approaches to leadership training 
programs, and their method of evaluation, will serve as a 
foundation and basis of comparison for the programs of the 
military services. This review is not intended to be all 
encompassing, but rather to single out individuals whose 
work addresses the major theme of this plan or has contributed 
significantly to the development of the military leadership 
programs. For a detailed discussion of the historical aspects 
of leadership research the reader is refered to Ralph M. 
Stogdill's Handbook of Leadership . 

B. GREAT MAN AND TRAIT THEORIES 

It was not very long ago that the subject of leadership 
training was considered an area not worth pursuing. The 
widely held belief was that the qualities of leadership were 
a fxinction of heredity, if one was not fortunate enough to 
be related to royalty there could be no expectation of holding 
a leadership position. This concept was formalized by Carlyle 
[1846] in his essay on heroes in which he proposed the 'Great 
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Man' theory of leadership. In 1960 Eugene E. Jennings pub- 
lished An Anatomy of Leadership: Princes, Heroes, and 

Supermen which reviewed and analyzed the research on this 
theory . 

With leaders so easily identifiable it was believed 
possible to determine what made the man/woman so great. 

This idea gave rise to the next major classification of 
leadership theory, the trait approach. Begining in the late 
1920 's studies were conducted to measure the personality and 
character of individuals who had reached positions of leader- 
ship. The focus of academic research remained fixed, 
throughout World War II, on defining those traits which sep- 
arated leaders from non- leaders. 

The implication of this theory would have been to screen 
leaders from non- leaders based on some identified leadership 
traits, and to provide training only to those individuals 
displaying those qualities. In actuality the war require- 
ment to increase manpower from "two hundred thousand in 1939 
to almost six million three years later" [Review of Education 
and Training for Officer, 1978, p. v.] resulted in a training 
program which presented the traits to all potential leaders. 

The research on trait theory revealed an interesting 
phenomenon, some of the studies Stogdill cites in his 1948 
survey contradict each other with respect to the traits 
associated with effective leadership. Accounting for these 
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discrepancies he adds situation to the general headings of 
capacity, achievement, responsibility, participation, and 
status as factors associated with leadership. In discussing 
the addition he writes: 

A person does not become a leader by virtue of the 
possession of some combination of traits, but the pattern 
of personal characteristics of the leader must bear some 
relevant relationship to the characteristics, activities, 
and goals of the followers . Thus leadership must be 
conceived in terms of the interaction of variables which 
are in constant flux and change. The factor of change 
is especially characteristic of the situation. .. It 
becomes clear that an adequate analysis of leadership 
involves not only a study of leaders, but also of 
situations . 



[Stogdill, 1974, p. 63] 

This urging of a situational view was far different 
from the pure situational theories of the time. The pure 
situationalist , e.g. Murphy Cl94l] , thought that leadership 
does not "reside in a person but is a fxinction of the occa- 
sion. The situation calls for certain types of action; the 
leader does not inject leadership, but is the instrumental 
factor through which a solution is achieved." [Stogdill, 1974, 
p. 18] 

What was suggested was a need to look at the interactive 
effects of both situational and personal factors. Researchers 
such as Westburgh [l93l] , Gibb [1954] , Stogdill and Shartle 
[1955] , Bennis [l96l] , and Hollander [1964] have elaborated 
on this interaction by proposing various sets of factors. In 
a 1974 survey of 163 additional studies, Stogdill pointed 
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out this need to consider both factors by listing some 
leadership characteristics in order to modify the pure sit- 
uational theorist's point of view. 

It is this combination of factors which today is commonly 

called the situational or interaction approach. Hersey and 

Blanchard summarize the approach with: 

The focus in the situational approach to leadership is 
on observed behavior, not on any hypothetical inborn or 
acquired ability or potential for leadership. The 
emphasis is on the behavior of leaders and their group 
members (followers) and various situations. 

[Hersey and Blanchard, 1977, 
p. 89J 

Within this major catagory are found many of the current 
theories or models of leadership that have either dealt with 
the issue of training or were influential in the design of 
the military's programs. A second group of theories which 
has had a like effect are those Stogdill [1974] calls 
"Humanistic" theories, those that are "concerned with the 
development of effective and cohesive organizations ." [Stogdill , 
1974, p. 21.] The work of McGregor, Blake and Mouton, and 
Likert fall within this catagory, and it is by this title that 
their works will be reviewed. Also reviewed are the studies 
by theorists in the Transactional, Transactional Analysis, 
and Transformational areas. 
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C. SITUATIONAL THEORIES 



1 . Contingency Theory 

Since 1951 Fred E. Fiedler and his associates have 
been studying, testing, and modifying the Contingency Model 
of leadership. To date it stands as the most researched of 
all models. While it is their training program. Leader Match, 
which is significant to this thesis, a background on the 
model is necessary in order to understand the training 
concept. 



The Contingency Model is based on two factors: (1) 
the motivational style of the leader; and, (2) the degree 
of situational control. 

The leader's motivational style is determined 
through the administration of an eighteen item bipolar ad- 
jective scale. This scale, the Least Preferred Co-worker 
Scale, is central to the model. Fiedler describes its oper- 
ation as: 

An individual who describes his or her least preferred 
coworker in very negative and rejecting terms (a low 
LPC) [task motivated] in effect shows a strong emotional 
reaction to people with whom he or she cannot work -- 
in effect, "if I can't work with you, you are no damn 
good!" This is the typical pattern of a person who, 
when forced to make the choice, opts first for getting 
on with the task and worries about his interpersonal 
relations later. Someone who describes even the least 
preferred coworker in relatively more positive terms in 
effect looks at the individual not only as a co-worker 
but also as a person who might otherwise have some 
acceptable traits. The high LPC leader [relationship 
motivated] sees close interpersonal relations as a re- 
quirement for task accomplishment. 

[Fiedler, 1977b, p. 199] 
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The situation is defined as "the degree to which 
the situation provides the leader with power and influence." 
[Csoka and Bons, 1978, p. 2953 It is measured on three 
scales: (1) Leader -- Member Relations, either good or poor 
it is the degree to which the leader is supported by the 
group; (2) Task Structure, how clearly the task is defined 
with respect to goals and procedures, this dimension being 
structured or unstructured; and (3) Position Power, strong 
or weak, it measures the degree to which the leader can 
reward and punish group members (subordinates) . When 
arranged on a continuum the situational control (originally 
called situational favorableness) forms eight octants as 
shown in figure 1. Also depicted in figure 1 is a plot of 
the results of Fiedler's studies. The horizontal axis rep- 
resents situational control while the vertical is a corre- 
lation between group performance and LPC score. 

Experiments conducted by Fiedler and others on groups 
ranging from field artillery NCO's to high school principles 
tended to show that "task motivated leaders perform generally 
best in favorable situations, i.e. under conditions in 
which their power, control, and influence are very high (or 
conversely, where uncertainty is very low) or where the 
situation is unfavorable, where they have low power, control, 
and influence. Relationship motivated leaders tend to per- 
form best in situations in which they have moderate power. 
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FIGURE I. HOW STYLE OF LEADERSHIP VARIES 
WITH SITUATIONAL CONTROL. 

SOURCE: adapted FROM LEADERSHIP a EFFECTIVE 

MANAGEMENT . Fr«d E. Fi«dl«r a Martin M. Chtmart, 1974. 



control, and influence.” [[Fiedler, 1976a, p. 354] 

There are two additional assumptions which are 
critical to the reported experiments with the model: (1) that 
leader attributes are stable over time; and, (2) that all of 
the possible factors which determine the situation can be 
summed into the favorableness, or unfavorableness, dimension. 
The number of variables being almost limitless in this latter 
factor . 

The contingency model has met with a great deal of 
criticism despite (or because of) the many experiments con- 
ducted to prove its validity. Hersey and Blanchard disagree 
with Fiedler's either/or position on the leader's motivation 
saying it is not sufficient that the situation vary from 
favorable to unfavorable, the orientation of the leader also 
may vary from task motivated at one extreme to relationship 
motivation on the other. They state that "Most evidence 
fe.g. the Ohio State Leadership Studies] indicates that 
leader behavior must be plotted on two separate axes rather 
than on a single continutom." [Hersey and Blanchard, 1977, 
p. 102] 

Stinson and Tracy [1974] raise three points based on 
experiments they conducted to verify the model. Briefly, 
they question: (1) the interpretation of the LPC scores, 

"By its structure the LPC is an attitudinal scale, but 
Fiedler has attempted to make inferences from it regarding 
the behavior or style of leaders." [Stinson and Tracy, 1974, 
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p. 4773; (2) stablity of the LPC score, "Fiedler's inter- 
pretation of the LPC score as an indicator of a stable 
personality trait is based on post hoc analysis and he has 
presented no independent evidence of the existence of this 
trait." |j). 478]; and (3) the post hoc measurement of situ- 
ational favorableness control. 

Ashour [1973] has criticized Fiedler's procedure for 
combining results of different octants together which "con- 
founds any predictive differences that might exist between 
octants .. .After applying Fisher's [l946] technique of combin- 
ing correlations and then analyzing the data, Ashour found 
that Fiedler's model failed the validity tests (i.e. non- 
significant correlations) in six of the eight situational 
octants'.' [Hendrix, 1976, p. 27] Ashour concluded therefore 
that the model contained serious theoretical and methodological 
flaws . 

The studies by Graen [1970, 1971] and comments by 
Holland [^1974] can be stimmed up with a Stinson and Tracy 
quote: "if the validity and reliability of the LPC score 

cannot be established with greater certainty, its use to 
provide data supporting a major leadership theory is inap- 
propriate." QStinson and Tracy, 1974, p. 485] 

Countering these criticisms Fiedler maintains that 
"these studies have methodological weaknesses which make them 
inadequate tests of the model." ^Fiedler, 1971, p. 202] 
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Specifically the Graen et al ^19713 study was cited as 
having poor manipulation of the power variable in its 
laboratory situation, a problem Fiedler believes he over- 
comes in his studies outside of the laboratory, e.g. ROTC, 

West Point, Belgian Navy, etc. The task structure manip- 
ulation and the distribution of the LPC scores among the 
octants Fiedler found as contributing to that study's 
weaknesses . 

Leaving the basic model and turning towards its 
implications for training recall the second assumption that 
any number of variables can go into determining situational 
control, training is such a variable. Fiedler maintains that 
a "leader's motivational orientation or personal style is 
likely to be relatively impervious to modification through 
short term training. On the other hand, training might 
improve the leader's potential influence and control (i.e. 
the situational control of the job)." CChemers and Rice, 1974, 
p. 1123 Training, in other words, is a modifier of the sit- 
uation. This, however, is not always good. 

Given a moderately favorable situation and a low 
LPC type leader, training in this situation can be predicted 
to improve the situation, thereby putting the low LPC leader 
in a favorable situation, which the model predicts as being 
the most advantageous for the leader and group performance. 

The same logic also applies to the high LPC leader found in 
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an unfavorable situation where training will hopefully raise 
the situation to one of moderate favorableness. The negative 
aspect comes when the low LPC leader in an unfavorable sit- 
uation moves into a moderately favorable situation where he 
loses the advantage. This Fiedler points out is why training 
does not always result in improved group performance. The 
implied course of action is for the leader to receive addi- 
tional training in order to move the situation to one of 
favorableness. When movement through training is not feasible 
Fiedler sugggests that assignment to leadership positions be 
based on the individual's motivation and the situation. A 
practice which, for all practical purposes, is impossible 
at lower command levels within the military, but, one which 
is being used for senior grade (general officer) assignments. 

This ability to change the situation through training 

was used by Fiedler, Chemers , and Mahr [19753 bo develop the 

Leader Match Training Program. The program involves a self- 

paced book entitled. Improving Leadership Effectiveness , 

which is designed to teach individuals how to apply the 

principles of the contingency model. 

In Part 1 of the manual, the individual completes the 
LPC scale, which indicates whether he or she is prima- 
rily concerned with interpersonal relations (high LPC) 
or with task accomplishment (low LPC). In Part 2, the 
individual learns how to measure situational control 
using scales designed for each of the three variables. 

Part 3, suggests methods for changing situational 
control and Part 4 provides guidance on how to help 
subordinate leaders improve their performance. 

[Fiedler and Mahr, 1979, p. 248] 
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It is the method of evaluation used for this program, 



advertised as a "fully validated training program," which 
is of most interest to this thesis. 

To examine the methods employed, four experiments will 
be reviewed in which the presentation of the Leader Match 
program reportedly led to significantly higher measures of 
leader effectiveness. These experiments, all involving 
military subjects, were conducted between 1977 and 1979. 

In the first experiment Navy officers and petty 
officers were randomly assigned to either a control or 
treatment group. Performance ratings were completed prior 
to training for both groups with no significant difference 
between groups reported. The treatment group then was given 
two, four-hour, training sessions in which they viewed a 
film on the theory, read the Leader Match manual, and part- 
icipated in a group discussion. Six months afterwards 
performance ratings were again gathered from at least two 
of the three original raters for each subject. The result, 
measured by mean performance change in rating score, showed 
"the difference between the trained and control groups' 
change scores .. .were highly significant." ^Leister et al., 

1977, p. 468j. 

Using the notation from Campbell and Stanley's Exper- 
imental and Quasi-Experimental Designs for Research on Teaching 
this experiment can be depicted as ; 
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Treatment Group: R 0^ X ©2 

Control Group : R 0^ ©2 

The symbols represent: R = Random Assignment; 0 = Observation; 

X = Treatment. The ordering from left to right signifies the 
passing of time, the last observation being O 2 . Thus, in 
this experiment the treatment measurement (O 2 - 0^^) > control 
(O2 - 03^). 

The second experiment, involving West Point cadets, 
follows the same design as the first but the treatment group 
was required to submit a written evaluation of the manual and 
a follow-up report three months later describing attempts to 
use the principles of Leader Match (Contingency Theory) and 
their relative success or failure. 

Each subjects' assignment to either the top, middle, 
or bottom third category (of all cadets at the school) based 
on leader effectiveness was the performance criterion selected. 
The result: "subjects in the experimental group were signif- 

icantly more in the top third and less in the bottom third 
categories than subjects in the control group." ^Csoka and 
Bons, 1978, p. 297] 

Experiment three differed in that a third control 
group was randomly selected and the treatment group was not 
required to provide feedback. This third group was told 
they were to be observed and evaluated during the time the 
experiment was being conducted. The purpose of this will 



36 



be discussed later. Battalion commanders were used to rank- 
order the subjects, again West Point cadets, based on dis- 
played ability to lead. These rankings were converted to 
decimal scores for comparison purposes (e.g. the first out 
of four was converted to .75; third out of eight was .62). 

An analysis of variance indicated that "student military 
leaders who were given the Leader Match training manual 
ranked significantly higher in comparative performance 
than those who did not use the manual ." CCs oka and Bons, 1978, 
p. 298] The informed control group performance ranking was 
below that of the control group which was not informed. The 
short-hand depiction of this experiment is: 

Treatment Group: R X 0 

Control Group 1: R 0 

Control Group 2: R (X) 0 

The (X) indicates a placebo treatment, in this case informing 
the group they were being observed. 

The final experiment. Army ROTC cadets at Advanced 
camp, involved a stratified random sample of schools grouped 
into upper, middle, and lower categories based on student 
perfomance at the previous year's camp. Three schools from 
each catagory were designated as treatment or control, a 
total of eighteen schools. The 190 cadets from the nine 
treatment schools were issued the manual and given a closed- 
book exam to insure that the training was completed. The 
subjects' performance at camp was based on two peer ratings. 
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advisor ratings, an orienteering exercise, and a tactical 
exercise. The latter two criteria did not show a significant 
difference between control and treatment groups while the 
"mean cadet performance of ROTC programs in which Leader 
Match was administered was significantly higher than that of 
control schools on measures directly related to leadership 
skills ."^Fiedler and Mahr, 1979, p. 251]. 

The purpose behind detailing these experiments is 
to call attention to the possible alternative hypotheses 
(explanation of group differences apart from those created 
by Leader Match) despite the popularity of the experimental 
design. In each experiment the possibility of sampling bias 
and rater bias are adequately addressed. The likelihood 
however, that the Hawthorne effect accounted for some of 
the difference is not so easily dismissed as Fiedler et al. 
would indicate. 

This major contaminant of evaluative methodology 
results from the subject having been singled out for special 
attention (e.g. training). In the first experiment the 
effect is rejected on the naive assumption that "training is 
a way of life in the military services ... participation in a 
12-hour training program is thus nothing out of the ordinary 
for most military personnel." C^eister et al. , 1977, p. 469] 
The fact is that 12-hours of concentrated leadership training 
is something special outside of a service school environment. 
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An item which is not reported in the case but one which would 
be significant to the impact the study would have on the in- 
dividual was whether or not the trainers were military or 
not. 

Further supporting their contention that the Hawthorne 
effect was negligable they argue that other training program 
experiments should also have reported positive results. This 
argument would be valid only after an analysis of the admin- 
istration of the experiments, other research efforts perhaps 
imposing other forms of control. 

In the last experiment the importance of motivation 
to the performance at ROTC advanced camp is minimized. They 
(Fiedler and Mahr) point out that there should have been a 
difference in the orienteering and tactical exercises if 
the training was responsible for no more than raising moti- 
vation. This justification is in error due to the relatively 
minor role motivation plays in successful performance on 
these exercises, knowledge of land navigation and tactics 
being the principal components of success, whereas the other 
performance measures are highly influenced by motivation as 
opposed to skill. 

Recognizing that the "method of choice for eliminating 
the Hawthorne effect is... the use of an alternative training 
program " [Fiedler and Mahr, 1979, p. 252] the researchers 
claim that the leadership instruction received by cadets 
during their previous year(s) of ROTC training constitutes 
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this alternative. Their claim fails to address a salient 
feature of the Hawthorne effect, that of being singled 
out for training. Across the board training does not meet 
this criteria. 

Csoka and Bons [1978] report that in their experi- 
ments (the second and third) the Hawthorne effect (as a 
confidence builder) is not easily dismissed. 

Having elaborated on the Contingency Model, Leader 
Match, and the validation studies of this training program, 
the point is all too clear. Even a leadership program 
as thoroughly researched as Fiedler's is subject to 
invalidities in the evaluation methodology. These difficul- 
ties lie at the heart of all assessment attempts and will 
serve as a useful comparison to the model proposed in this 
thesis . 



Prior to moving on to the other situational 
theorists, the following is offered as testimony to Fiedler's 
work: 

Fiedler's model might best be characterized as a model 
of controversy, and perhaps the birth of this contro- 
versy is Fiedler's greatest contribution to the under- 
standing of leadership. The introduction of his model 
has resulted in many researchers attempting to test the 
model, and each test has perhaps added a little to the 
knowledge of leadership. 

[Hendrix, 1976, p. 27] 
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2 . Situational Leadership 



Hersey and Blanchard, like Fiedler, maintain that 
there is no one best style of leadership. They differ in 
their contention that the needs of the group and the envir- 
onment make the successful leaders adapt their behavior to 
the situation. 

The Situational Leadership theory (previously called 
Life Cycle Theory of Leadership) starts with a two dimensional 
grid, the vertical axis representing relationship behavior 
while the horizontal axis signifies task behavior. They 
define these variables as: 

Task Behavior - The extent to which leaders are likely 
to organize and define the roles of the members of their 
group (followers ) ; to explain what activities each is to 
do and when, where, and how tasks are to be accomplished; 
characterized by endeavoring to establish well-defined 
patterns of organization, channels of communication, and 
ways of getting jobs accomplished. 

Relationship Behavior - The extent to which leaders are 
likely to maintain personal relationships between them- 
selves and members of their group (followers) by opening 
up channels of communication, providing socio-emotional 
support, "psychological strokes," and facilitating 
behaviors . 



^Hersey and Blanchard, 1977, 
p. 104] 

It is based on a curvilinear relationship between 
task and relationship behaviors and maturity of their 
followers for a specific task. Figure 2 depicts this relation- 
ship . 

Maturity of the follower is defined as "the capacity 
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to set high but attainable goals (achievement - motiva- 
tion), willingness and ability to take responsibility, and 
education and/or experience of an individual or a group.” 
[^Hersey and Blanchard, 1969, p. 28] This concept of the 
follower being a vital part of a leadership theory is signif- 
icant in considering an evaluation of any training program. 
Stogdill writes, "Change in the leader is significant only 
if it produces an impact on the follower group.” Cl974, p. 4221 

According to the theory, as the maturity level of the 
individual /group rises (moves from right to left) the approp- 
riate behavior or leadership style should change from high 
task and low relationship to one of high task/high relation- 
ship. At the midpoint in the maturity continuum the curve 
slopes downward to the left as maturity continues to increase, 
this results in a decrease in the relationship behavior as 
the group passes into what Maslow termed the esteem and self- 
actualization needs, they provide their own relationship 
needs or "strokes.” Throughout this change in relationship, 
task behavior has continued to decline. 

In effect this theory says that given a certain task 
for which the followers display high maturity a low relation- 
ship/low task behavior style is appropriate. This point, as 
will be seen, corresponds to Blake and Mouton's Impoverished 
leadership style which the Grid model would never consider 
appropriate. It should be noted that this point corresponds 
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with Maslow's self-actualization need which he maintains is 
reached by only a very few individuals. So while the leader's 
behavior at this point may be correct, so few of the fol- 
lowers will ever reach this maturity level that the behavior 
will usually never be seen. The theory is consistent with 
McGregor's Theory Y which allows for a wide range of style, 
depending on the work group and the situation. 

Hersey and Blanchard have sectionalized the curve 
according to the quadrant it falls in and have termed the 
leadership styles as: Telling (high task/low relationship); 
Selling (high task/high relationship) ; Participating (low 
task/high relationship); and Delegating (low task/low rela- 
tionship) . These terms describe the style appropriate to 
the level of maturity. 

While noting that "changing the style of leaders is 
a difficult process, and one that takes considerable time to 
accomplish" Hersey and Blanchard, [1977, p. 150] have devel- 
oped a test to measure the leader's self perception of his/ 
her style, style range, and style adaptability (resulting 
profiles from the instrument) . This instrument is the Leader 
Effectiveness and Adaptability Description (LEAD) -- Self. 

It consists of twelve situations of varying follower maturity, 
three from each quadrant. Each situation is followed with 
four possible responses from the leader. Based on the re- 
sponse given, the leader is classified into a certain style. 
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The consequence of this classification is an ability to 
recognize which level of management the leader's style is 
most appropriate for and structure future training and 
development courses to expand the leader's adaptability. 
Hersey and Blanchard report that most leaders are able to 
modify their style after some training in Situational 
Leadership theory. 

To assist the leader in determining what training 
may be needed there is also a LEAD -- Other instr\iment 
which the leader's peers, superiors, and subordinates 
complete on the leader. The test is identical to the LEAD - 
Self in the situations but is written in the third person. 

Despite the current popularity of the Situational 
Leadership model within the military and civilian communi- 
ties (e.g. Xerox Corporation) it is not without its critics. 
Barrow [19763 criticized both the theory and instrument as 
not having been empirically tested, a comment which is still 
voiced. Information concerning the testing of the model is 
very limited. Due to its commercial orientation little has 
been published in academic journals concerning measures of 
its validity, thereby restricting the type of criticism 
found in the Contingency Model. 

Given the overall influence of Hersey and Blanchard' 
model it is appropriate to consider the model's implication 
with respect to a major leadership function, decision making 
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The Vrooa and Yetton aodel considers the situational aspects 
of the problen: at hand in prescribing the suitable leader- 
ship style. 

3 . KorrLative liodel of Leadership Styles 

The objective of the Vrooa and Yetton aodel is to 
prescribe an appropriate leadership style in making decisions. 
It is formulated on a set of five ass»japtions ; (1) That the 
model be operational (leader required behaviors are unambig- 



uous) ; (2) Tnat the degree 
problem solving can vary; 
is applicable to all sit'.:a 
specific problem contains 
That the leadership method 
should not constrain the m 



of subordinate participation in 
(3) That no one leadership method 
tions; (4) That the issues of a 
the key to its solution; and (5) 
used in response to one sit^iation 
ethod or style used in other 



sLtiuations . 



Based on tr;ese ass’rmptions two models were developed, 
one for group, the other for individ’ral problems (i.e. the 
number of subordinates affected by the decision, one or 
many). Each model is comprised of tr.ree variables; the 



attributes of the specific pr 
the carnality and acceptance of 
possible leadership styles. 

There are seven basic 
tions , vhich require a yes or 
called for eight plus tvo sup 
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FIGURE 3. NORMATIVE MODEL FLOWCHART 

SOURCE' LEADERSHIP AND OECISION 

MAKING . Victor H. Vroom a Philip W. Yclton, 
l«73. 



and acceptance priority. 



As a by-product of their research Vroom and Yetton 



have developed a program of leadership development. The 
purpose of course to "encourage people to examine critically 
the leadership methods they use in concrete situations in 
order to better fit their 'style' to the situational demands." 
[Vroom and Yetton, 1973, p. 2083 Their program, taking from 
15 to 22 hours for the first six phases, consists of seven 
phases in total: (1) Training in recognizing the differ- 

ences in own and other's decision process; (2) Diagnosis 
of one's own leadership style; (3) Practice in using deci- 
sion processes; (4) Understanding the consequences of 
different decision processes; (5) Training in the Normative 
Model; (6) Feedback based on behavior on the standardized 
problems; and (7) Follow-up. The feedback package in 
phase six comes in the form of a computer printout showing 
the leader's responses in comparison to the other leaders in 
the group and agreement with the model's predicted solutions. 



were unaware of the Normative model were asked to describe 
a problem, its attributes, and the relative effectiveness 
of the solution. The correlation between what the model would 
predict to be the most effective course of action and what 
course the manager reportedly took to arrive at the decision 
was sought to verify its overall validity. The results were 




In attempting to validate the theory, managers who 
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"all in a direction consistent with the hypothesis that the 
model has a higher ’probability’ of generating decisions of 
high quality and acceptance, but none reach conventional 
levels of statistical significance." [Vroom and Yetton, 1973, 
p. 183] The most recent work on the model by Jago and Vroom 
[1978] centers on validating the problem sets developed for 
the training and testing their predictive ability. 

In a recent article on the Vroom and Yetton model 
Field [1979] critiques several features of the theory. First 
is the possiblity of error due to the method of obtaining 
self-reports from the managers on decisions they have made 
in the past, "the manager will likely report high decision 
effectiveness, quality, and acceptance and may have cogni- 
tively distorted the actual levels of these variables" 

[p. 252], He concludes his remarks with "The utility of the 
Vroom - Yetton model is questionable for two reasons. First, 
it is not as parsimonious as other models of leader decision 
process choice [it is overly complex^. Second, it deals with 
only one aspect of leader behavior, that of selecting dif- 
ferent decision processes for different problem situations" 
[p. 256]. 

This completes the review of the situational models 
of leadership. While not exhaustive of the field, it does 
provide insight into the major theories. In the next section 
the theories which maintain that there is one best style of 
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leadership are reviewed. 



D. HUMANISTIC THEORIES 
1. Managerial Grid 

Hersey and Blanchard's comment on the contingency 
model's failure to recognize that leadership must be measured 
on two axes stems from the results of the Ohio State Leader- 
ship studies. These studies, begun in 1945, sought to iden- 
tify various dimensions of leader behavior, that is, the 
behavior of an individual when directing the activities of 
a group toward some final goal. Initially arriving at four 
dimensions, these were later reduced to two, Consideration 
and Initiating Structure. Consideration is the "behavior 
indicative of friendship, mutual trust, respect, and warmth 
in the relationship between leader and members of his staff" 
[Hersey and Blanchard, 1977, p. 94]. Initiating Structure 
refers to "the leader's behavior in delineating the relation- 
ship between himself and members of the work group and in 
endeavoring to establish well-defined patterns of organi- 
zation, channels of communication, and methods of procedure" 
[Hersey and Blanchard, 1977, p. 94]. Both dimensions being 
expressed in behavioral terms, i.e. how the leader acts. 

Following in this view, Robert R. Blake and Jane 
S. Mouton constructed a similar framework using concern for 
people and concern for production. Analogous to Consider- 
ation and Initiating Structure respectively, the major 



51 



difference is that the use of the term concern makes the 
dimension attitudinal vice behavioral. This model, the 
Managerial Grid, has been widely used as a leadership 
training tool in both the civilian and military programs to 
get the leader to recognize the interdependence of the two 
dimensions . 

The grid itself is constructed with concern for 
people along a nine-point vertical axis and production con- 
cern along the nine point horizontal axis. Of the 81 possible 
points of interaction, five have been identified as being 
primary styles, they are depicted on figure 4 and defined 
as : 

Impoverished (1,1). Exertion of minimum effort to get 
required work done is appropriate to sustain organiza- 
tional membership. 

Country Club (1,9). Thoughtful attention to needs of 
people for satisfying relationships leads to a comfort- 
able friendly organization atmosphere and work tempo . 

Task (9,1). Efficiency in operations results from 
arranging conditions of work in such a way that 
human elements interfere to a minimtim degree. 

Middle-of- the-Road (5,5). Adequate organization per- 
formance is possible through balancing the necessity to 
get the work out while maintaining morale of people 
at a satisfactory level. 

Team (9,9). Work accomplishment is from committed 
people, interdependence through a "common stake" in 
organization purpose leads to relationships of trust 
and respect. 

QHersey and Blanchard, 1977, p. 963 
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FIGURE 4. MANAGERIAL GRID 

SOURCE: ORGANIZATIONAL 

BEHAVIOR: UTILIZING HUMAN RESOURCES. Paul 
Hartay ft Ktnnath H. Blonchard, 1977, p. 96. 
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In designing leadership training around Managerial 
Grid (also applicable to organizational development, inter- 
group relations, etc.) Blake and Mouton have constructed a 
six phase program. 

Starting with the leader recognizing his/her own 
style of leadership and culminating with the attainment of 
some organizational goals, it covers both individual and 
organizational leadership. An unique aspect of the Grid 
is that it is taught to the leaders /managers of an organ- 
ization by personnel from its own ranks. These instructors 
or facilitators having been trained and certified in the 
conduct of the program. While the length of the phases may 
run from one week to several years, the major objectives of 
each phase are: 

Phase I. Management Development. To increase the leader's 
awareness of own attitudes, and to familiarize him/her with 
the Grid system so that it may be applied in analyzing 
own and organizational leadership. 

Phase II. Team Development. Analysis of own work group 
environment is conducted to determine what, if any, 
barriers to effective leadership exist. Team provides 
feedback to leader on his/her style so that the leader may 
analyze the gap between the 9,9 style leader (where the 
theory says he should be) and where he and others 
perceive him to be. A "standard of excellence" is estab- 
lished for individuals and groups' performance. 

Phase III. Intergroup Development. Similar to the 
second phase, the emphasis is on organizational leader- 
ship at the interface of various work groups. Barriers 
in planning and communicating between the groups are 
identified and resolved. 

Phase IV. Organizational Goal Setting. Leaders use 
skills obtained in previous phases to objectively. 



54 



logically, and explicitly define the organizations goals. 

Phase V. Implementation. Implement Phase IV plans. May 
take several years . 

Phase VI. Stabilization. Measurement and evaluation of 
success in reaching organizational goals . 

Throughout the program, and in the final analysis, 
questionnaires, forms, etc., are provided to the organization 
to determine its (the Grid's) effectiveness. 

Again, it is the final phase, evaluation, which this 
thesis is primarily concerned with. Despite its popularity 
"surprisingly few areas of the Grid have been subjected to 
rigorous research" [^Huse, 1975, p. 160]. Holding much the 
same opinion as Odiome [1964] that training must result in 
an increase in tangible economic indicators to determine its 
effectiveness, Blake and Mouton hold that within a profit 
making organization the ultimate measure is financial. 

In one widely publicized evaluation of a 4,000 member 
division of a petroleum corporation, an increase in profits 
and productivity and a decrease in controlable costs were 
pointed to as evidence of the success of the Grid training. 
The problem with the evaluation was that many of the indi- 
cators, or criterion, changed as a result of; (1) organiza- 
tional impact from the parent headquarters in the form of 
forced reduction in the size of the work force; (2) failure 
to establish a control group, and (3) no measure of the 
possible Hawthorne effect. The design of the program (in 
retrospect) is most like that of Campbell and Stanley's 
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one-shot case study and one-group pretest -- posttest designs 
(the two most susceptible to sources of invalidity) . These 
problems are frequent in the studies of the Grid; Blake and 
Mouton Beer and Kleisath [196^ , Beckhard (|l965l , and 

Kuriloff and Atkins CL96^ . Stogdill, citing the same petro- 
leum study stated "The control in these studies are less than 
adequate. The research was conducted during a long economic 
upswing when, in general, businesses were increasing their 
profits" C1974, p. 1893. 

It is unfortunate that a program for leadership 
training as developed and as widely used as the Grid should 
suffer from such poor evaluative techniques. As discussed 
in the introduction, the military services, which have 
taught Managerial Grid extensively (e.g. US Army Management 
School, 1964), can no longer afford to adopt a program which 
has nothing more to offer than intuitive appeal. Furthermore, 
the commitment called for on the part of the subordinate 
cannot be developed in a combat environment through "delib- 
eration and debate around major organization issues and 
by the mutual identification of sound objectives" [Pitts, 1976, 
P. 573. 

Blake and Mouton are not alone in arguing for a one 
best style of leadership. Another theory which has become a 
cornerstone in leadership research and training are McGregor's 
theories . 
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2 . Theory X and Theory Y 

A classic theory both in terms of its use in military 
instruction and impact on behavioral research is McGregor's 
concept of Theory X and Theory Y. These two theories dichot- 
omize the beliefs or assumptions one may have about human 
nature and behavior which formulate an individuals leadership 
style . 

The assumptions of Theory X include: (1) the average 
human being has an inherent dislike of work and will avoid 
it if possible; (2) most people must be coerced, controlled, 
directed, or otherwise threatened with punishment to get 
them to put forth adequate effort toward the achievement of 
organizational objectives; (3) The average human being prefers 
to be directed, wishes to avoid responsibility, has relatively 
little ambition, and wants security above all else. 

A leader ascribing to these beliefs would tend to 
rationalize poor organizational performance by blaming the 
people with which he/she must operate. The leader is more J| 
likely to control or supervise his subordinates much more 
closely through the use of his/her authority. 

On the other hand. Theory Y assumes: (1) the expend- 
iture of physical and mental effort in work is as natural as 
play or rest; (2) external control, and the threat of punish- 
ment are not the only means for bringing about effort toward 
organizational objectives; (3) commitment to objectives is a 
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fianction of the rewards associated with their achievement; 

(4) the average htiman being learns, under proper conditions, 
not only to accept but to seek responsibility, and (5) the 
capacity to exercise a relatively high degree of imagination, 
ingenuity, and creativity in the solution of organizational 
problems is widely, not narrowly, distributed in the popula- 
tion. It is this assumption which gives credibility to the 
Vroom and Yetton Normative model for the high acceptance 
problem requirements. 

The supervisor whose asstimptions about htiman nature 
are similar to the Theory Y principles has greater flexi- 
bility to operate over a range of leadership styles. He or she is 
able to determine the appropriateness of exercising his/her 
authority to control the subordinates when a lack of commit- 
ment to the organizations goals fails to stimulate the 
necessary self-control and self-direction to insure efficiency. 
This principle of integration, creating a condition in which 
subordinates can realize their own goals by working towards 
the organizations, is central to Theory Y. 

Above all else, it must be remembered that Theory Y 
does not imply permissive leadership, a common misunderstanding 
of McGregor's work. It allows for growth of the organization, 
unencumbered by the resource requirements devoted to the 
control of employees otherwise willing to support the organi- 
zations goals, but not trusted by the leader with the Theory 
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X assumptions. 

To aid the leader in moving toward a Theory Y 
belief it was proposed that training in the principles of 
both X and Y would make it possible for leaders to "evaluate 
their own actual situation to determine the extent to which 
it approximates the assumptions contained in Theory X, as 
contrasted with those in Theory Y. Once the theory-versus- 
practice analysis has been completed, causes for the differ- 
ences are identified" I^Blake and Mouton, 1978, p. 78]. 

McGregor's theories have had substantial impact on 
the works of other behavioral scientists and their contribution 
to the study of leadership in general and military leadership 
specifically. Renis Likert is one such researcher. 

3 . Management Systems 

Perhaps best known in the military community for 
his influence on the Navy's Hiaman Resource Management Support 
System, Renis Likert's work at the Institute for Social 
Research, University of Michigan, has emphasized the leaders' 
need to consider the human contributions to an organization's 
output. His efforts over the last four decades have centered 
on identifying the leaders role in the organization, his/her 
influence as a leader, the organizational characteristics 
associated with various leadership styles, and the assignment 
of costs to the human asset. 

The first effort, the linking-pin concept, identifies 
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the leader as a member of two groups, subordinate in one, 
superior in another. Within both he or she must be perceived 
by the other members as a part of their group in order to 
exert any influence and be effective. This second concept 
has been termed the interaction -- influence principle. 
Stouffer et al., wrote of this principle in The American 
Soldier [1949] : 

If an intermediate- level officer is to be a real leader, 
he has a dual role to play. He must accept the norms 
and values of superior authority, thus serving as an 
agent of the impersonal and coercive organization of which 
he is part. To the extent that he does this effectively, 
his superiors regard him highly. At the same time, he 
must win the willing followership of the men tinder him, 
so that he welds over them authority which they themselves 
have given him. 



[Gibb, 1969, p. 236] 

Two variables are at work in this principle. First, 
the more influence a leader has on his superiors the greater 
will be his influence with the subordinates. Second, "the 
more that managers allow themselves to be influenced by their 
subordinates, the more influence they, in turn, exert on 
subordinates" [Rush, 1976, p. 8-8]. Combined, these variables 
spell out a leadership style resembling Theory Y or partic- 
ipative which has been termed a supportive relationship. 

In many studies of organizations Likert found that 
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the participative style was only one of four types, or 

systems, possible. Arranged in order from that most commonly 

associated with a Theory X asstimption, these systems are; 

System 1 (Exploitive -- Authoritative). No confidence 
or trust in subordinates. Top down orientation of decision 
making, highly centralized management. Fear and punish- 
ment are typically employed to force greater production. 

System 2 (Benevolent -- Authoritative) . Condescending 
attitude of managers towards subordinates. Some dele- 
gation of decision making from top of organization. 

System 3 (Consultative). Substantial, but not complete, 
trust in subordinates. Strategic decisions remain at 
top, day-to-day decisions made at lower levels. Rewards 
primarily used to motivate workers. 

System 4 (Participative -- Group) . Complete trust and 
confidence in subordinates. Good communications in 
both directions allows for decentralized approach to 
decision making. Intrinsic rewards motivate employees. 

[Hersev and Blanchard, 1977, 
p. 73 

It is towards this fourth system that Likert says 
all organizations should strive. To help the company or 
command to move in this direction an instrument was developed 
which monitors the climate of the organization on eight char- 
acteristics; leadership, motivation, communication, decision 
making, interaction and influence, goal setting, and control 
nrocess . 



The Navy has been influenced by Likert's (as well as 
Bowers and Seashore's) work in designing its 88-item Human 
Resource Management Survey. The dimensions measured include; 
command climate, supervisory leadership, peer leadership, and 
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work group processes. Seventeen separate indexes comprise 
these four dimensions with an additional eight for emphasis 
and end result measures . 

The results of the survey administration can aid the 
leader in monitoring his/her own style and the degree to 
which subordinates perceive him/her as a member of the group 
(ability to influence). 

Likert's later work in human resource accounting has 
evoked considerable interest, but limited acceptance and is 
not within the scope of this study. 

As in the case of the situational leadership 
theorists, the scope of literature reviewed in this section 
is not all inclusive. The models selected for inclusion have 
provided the reader with a sufficient background on the body 
of research which espouses a one best style of leadership. 

In the following section the interaction of leader and fol- 
lower will be reviewed from a slightly different perspective. 

E. TRANSACTIONAL AND TRANSACTIONAL ANALYSIS THEORIES 
1. Transactional 

Transactional leadership, similar to the concept of 

the exchange between organizations and members (March and 

Simon, 1958), refers to the transaction between leader and 

follower. T.O. Jacobs [1971] provides this explanation; 

Leaders act as organization representative by providing 
earned benefits to their followers, while at the same 
time guiding them toward satisfaction of organizational 
goal attainment needs. There is substantial evidence 
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supporting the view that such transactions do underlie 
organizational membership, and that both the organiza- 
tion and the members have expectations that must be met 
in order for the exchange to be considered fair. 

[Jacobs, 1971, p. 96] 

As a consequence of failing to meet their expec- 
tations either side will become dissatisfied and seek to 
terminate the relationship or modify it so that an equitable 
exchange is achieved. One of the critical expectations on 
the part of both groups is reward for benefits provided, the 
group expects certain behavior from its leader which it rewards 
with esteem, influence, etc. The leader, on the other hand, 
expects productivity from the group which he/ she rewards 
through contributions to the group ' s/ individual ' s needs 
(physiological through self-actualization) . Jacobs discusses 
in detail the implications surrounding expectations, percep- 
tion of rewards, success or failure of the group, and other 
aspects of this social exchange view of transactional leader- 
ship . 

Hollander [1978] provides this view of transactional 
leadership ; 

It is necessary to look at the leader - follower relation- 
ship, and not only at the leader. A fuller view of 
leadership needs to include followers and their responses 
to the leader. This process forms the basis for the 
transactional perspective, or approach to leadership. 

[Hollander, 1978, p. l] 

In outlining the features of this perspective, 
Hollander, emphasizes a "fair exchange" of benefits and rewards 
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between follower and leader. His description of the exchange 
theory is very similar to Jacobs' and need not be detailed 
here . 



James MacGregor Burns, in contrasting transactional 
and transforming leadership, speaks of transactional leader- 
ship as: 

Occur [ringj when one person takes the initiative in making 
contact with others for the purpose of an exchange of 
valued things ... Each party to the bargain is conscious of 
the power resources and attitudes of the other. Each 
person recognizes the other as a person. The bargainers 
have no enduring purpose that holds them together; hence 
they go their separate ways. A leadership act took place, 
but it was not one that binds leader and follower together 
in mutual and continuing pursuit of a higher purpose. 

[Bums, 1978, p. 19] 

He further contends that it is the nature of trans- 
actional relationships to be short-lived, both leader and 
led having to move on as needs are fulfilled. "Most important, 
the transactional gratification itself may be a superficial 
and trivial one" QBurns, 1978, p. 258]. 

2 . Transactional Analysis 

Developed by Eric Berne [1964] and popularized in 
works by Thomas Harris [1969], Transactional Analysis (TA) 
is a method for analyzing the behavior exhibited in an exchange 
between people. 

At the center of the model lies three ego states; 
Parent, the evaluative, rule enforcing state; Adult, the 
rational, decision making state; and. Child, the emotional 
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state. In the healthy individual a balance of all three 
states exists while in the unhealthy person any one state 
may dominate. 

The transaction between two people is analyzed as a 
function of which state they are "dealing” from and to 
(stimulus/response) . 

The key to continued communication is parallel trans- 
actions. This implies, for example, a Parent addressing 
Child as stimulus would receive Child addressing Parent 
response. If the transaction is crossed, e.g. Adult to 
Adult stimulus receives a Child to Parent response, commun- 
ications breaks down between the individuals . 

TA analyzes both one-time and continuing transactions 
with a view toward identifying roles, working toward estab- 
lishing parallel transactions, and reducing the games people 
play in their interrelationships. A game is a transaction 
in which the person is not being honest, some ulterior motive 
is driving the transaction. 

Criticized as being too simplistic, the support and 
attention given TA in the last few years stems from this sim- 
plicity. 

In 1973 the US Army Chaplain Corps, in response to 
a request from the Deputy Chief of Staff for Personnel, devel- 
oped the Personal Effectiveness Training course (PET) . PET 
used the principles of transactional analysis as developed by 
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Beme and Harris. Its primary focus was to provide training 
in communication and personal counseling. While the course 
no longer exists in its entirety, the concept is frequently 
discussed in unit and service school training. 

Transactional and transactional analysis theories 
deal with the relationship between follower and leader as 
the result of some exchange. In the next section this 
exchange or expectation will be contrasted to the view 
expressed by Bums. 



F. TRANSFORMATIONAL LEADERSHIP ^ 

Transformational leadership, as defined by James 

MacGregor Burns in his book Leadership , is: 

Transforming leadership, while more complex [than trans- 
actional] , is more potent. The transforming leader recog- 
nizes and exploits an existing need or demand of a potential 
follower. But, beyond that, the transforming leader looks 
for potential motives in followers, seeks to satisfy higher 
needs, and engages the full person of the follower. The 
result of transforming leadership is a relationship of 
mutual stimulation and evaluation that converts followers 
into leaders and may convert leaders into moral giants. 

[Burns, 1978, p. 4] 

The key to Bums' theory is moral leadership which 
"emerges from, and always returns to the fundamental wants 
and needs, aspirations, and values of the followers" [p. 4] . 

A political scientist, he draws from the experiences of 
leaders throughout history for examples. He defines leader- 
ship as : 
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A function of complex biological, social, cognitive, and 
affective processes, that is closely influenced by the 
structures of opportunity and closure around it, that may 
emerge at different stages in different peoples' lives, 
that manifests itself in a variety of processes and 
arenas - in short, we have seen that the usual general- 
izations [traits, leaders being bom and not made and 
vice versa] are without foundation. 

[Bums, 1978, p. 427] 

Philosophical in nature, his arguments are not readily 
molded into the experimental design features for validity 
and reliability testing which seem the prerequisite for 
military training. What then is the value in including his 
theory? Simply put, it provides the stimulus needed to 
look at leadership from other than the behavioral scientists 
point of view. This refreshing insight will undoubtedly 
leave its impact on the armed services in the near future. 



G. SUMMARY 

The preceding literature review has provided the reader 
with a general orientation of those theories which have 
contributed significantly to the lander standing of leadership. 
Despite the differences in perspective with regard to leader- 
ship theory and the characteristics of an effective leader 
development program, certain commonalities are found. Foremost 
is the role of the followers, their influence on the leader 
and reaction to his/her attempts to influence are paramount 
in determining the effectiveness of the leader and subsequently 
the success or failure of the organization. 
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The second major point of agreement, with the single 
exception of Fiedler, is that the leader is capable of 
modifying his/her style, behavior, attitude towards the 
follower. This change is contingent upon the effectiveness 
of the change agent, in this case training. Again Fiedler 
is heard as the one dissenting voice vis-a-vis other theorists 
on the efficacy of leadership training. 

It is postulated that the ability of any training program 
to produce better leaders can only be assessed in light of 
its relative merits and demonstrated effect on the organ- 
ization and individual leader. Broad generalizations about 
the capabilities of training to bring about change are inap- 
propriate in this context. 

In the following chapter the leadership training programs 
developed by each service are examined in light of the 
theoretical concepts reviewed here. Particular attention 
is focused on the environment in which the US Army's 
program, and consequently its evaluation, must operate. 
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III. LEADER DEVELOPMENT IN THE MILITARY 



As stated in the introduction, the study of leader 
development is most appropriately considered in the context 
of systems presently in place. The systems applicable to 
this thesis, apart from the leadership training program 
itself, include the service's organization for training and 
the personnel management system. In this chapter these 
systems are reviewed for each service as a prelude to 
detailing their approach to leadership training. 

In order to make the task of defining and comparing 
the various personnel systems manageable, only that portion 
of the system applicable to officers is studied. 

A. UNITED STATES ARMY 

1 . Organization for Training 

During the 1970 's considerable change was seen in the 
organization and operation of the US Army. Significant among 
those areas affected was military training. This section 
details the current organizational structure as it pertains 
to the areas of policy, doctrine, research, and implementation 
of leadership training. 

In 1973 a major Army-wide reorganization resulted in 
the creation of the Leadership Branch, Human Resources Devel- 
opment Directorate, Office of the Deputy Chief of Staff for 
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Personnel (ODCSPER) . The function of this office is primarily 
one of monitoring the leadership issues in the Army and form- 
ulating policy. 

Another organization created in 1973 is the US Army 
Training and Doctrine Command (TRADOC) . As the name implies 
TRADOC is the proponent agency for developing, managing, and 
supervising the training of individual officers and enlisted 
personnel throughout the Army. It commands all Army service 
schools with the exception of the Army War College and the 
Military Academy. 

As discussed in the introduction, the initial emphasis 
in TRADOC centered on the technical competence required for 
combat. This orientation first focused on the institutional 
training, i.e. service schools. Gradually, as it was realized 
that the majority of learning took place at the unit level, a 
decentralized approach was adopted. At this stage exportable 
training packages were developed for both the individual 
soldiers and units. 

Under this decentralization the various service schools 
became the primary source of Army doctrine and training devel- 
opment. Each school evaluates proposed doctrinal changes 
within their area of expertise and forwards their recommen- 
dations to TRADOC for approval and implementation. In de- 
signing specific courses (such as the initial entry Officer 
Basic Course) the school incorporates its specialty training 
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with materials from other areas, proponency for which has 
been given to another school. A great deal of latitude is 
given each school in designing its courses. This has led to 
a wide variation in the approach taken toward the common 
military subjects, i.e. leadership. The result is that no two 
service schools approach the subject of leadership training 
with the same material or in the same manner. 

Compotinding this problem is a lack of a clear under- 
standing concerning the proponency of leadership research, 
training development, and doctrine. Within the area of 
leadership training there are a number of agencies reporting 
to TRADOC. The US Army Infantry School (USAIS) is designated 
as proponent for the development of concepts and doctrine for 
military leadership. The Commander, US Army Administration 
Center (ADMINCEN) is tasked as "the proponent for the devel- 
opment of concepts and doctrine related to personnel manage- 
ment to include human resources development” [^TRADOC Regulation 
600 - 33 - Army Regulation 310 - 25 , the dictionary of Army terms, 
includes leadership within the definition of "human resources 
development ." 

Other TRADOC agencies involved in leadership develop- 
ment include the Command and General Staff College which is 
primarily concerned with leadership battle roles and the 
Organizational Effectiveness Center and School which has re- 
cently received proponency for common tasks in the areas of; 
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communications, human relations, counseling, supervision, 
decision making, management sciences, planning, and ethics. 
These are also the areas identified in the organizational 
leadership model which will be discussed later. 

Efforts within TRADOC to clarify proponency have 
centered on differentiating organizational leadership 
(ADMINCEN) from individual leadership (USAIS) , though this 
distinction was never clearly specified. 

Outside of the TRADOC chain of command the US Army 
War College and the Military Academy are both involved in 
leadership research and training development as it pertains 
to their particular curricula. 

Within the area of leadership research the Army 
established a contract in 1951 with George VJashington Univer- 
sity for the establishment of a Human Resources Research 
Office. This office served as the Army's "principal re- 
search and development agency in the field of training methods 
training content, and training evaluation" [[Lavisky, 1977, 
p. 107]. This contract was terminated in 1969 and the agency 
was separately incorporated as the Htiman Resources Research 
Organization (HumRRO) , This new organization, comprised of 
many of the people who had previously worked for the univer- 
sity, provided the Army with a resource for "conducting 
studies and research and development on training, needs for 
training devices, motivation, and leadership" QUavisky, 1977, 
p. 109]. 
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During the 1973 reorganization of the Army the 
special contract with HumRRO was terminated and the training 
research integrated into an in-house organization. This in- 
house agency eventually came to be called the Army Research 
Institute for the Behavioral Sciences (ARI) and was assigned 
the mission of "conducting the Army's research and develop- 
ment program in human performance, military selection, behav- 
ioral evaluation, motivation, race relations, behavioral and 
social aspects of drug abuse, social change, soldier- community 
relations, career incentives, and training" [Lavisky, 1977, 
p. 152] . HumRRO still exists as a private corporation and 
does the bulk of the research work contracted out by ARI, 

Because so many agencies were involved in some phase 
of leader development, the situation the Army found itself 
in as it entered the 1980 's was one where: 

Leader research, concept development, and training devel- 
opment activities have been carried on unsystematically 
and with less than resource/cost effectiveness. As such 
there are undesignated proponencies ; fragmented propon- 
encies; multiple and shared proponencies for the same 
area; developmental efforts being conducted in semi- 
isolation; and disconnects between researchers, combat 
developers, and training developers. 

[DA, ADMINCEN, 1979, p. 26] 

To address this problem an Army Leadership Action 
Planning Conference was convened in February 1980. Made up 
of representatives from all the agencies described above, 
the purpose of the conference was to draft a comprehensive 
plan to guide the Army's leadership training and development 
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activities. The attendees identified the following objective 
in the areas of policy, coordination, and integration of 
effort : 

Develop and implement policies and procedures that empha- 
size centralized planning and coordination and promote 
decentralized execution to support leader development. 

[Leader Development Plan, 

April 1980, p. 6] 

Tasks and responsibilities designed to achieve this 
objective were also spelled out. The Deputy Chief of Staff 
for Personnel office is to prepare and coordinate an Army 
regulation on leader development that prescribes objectives, 
responsibilities, and administrative procedures by September 
1981. TRADOC, by October 1980, is to designate a single 
integrating agency for leader development . 

While it is far too early to judge the effectiveness 
of the conference, indications that the recommendations will 
be adopted are positive. 

In the next section the second major system impacting 
on leader development, the personnel management system, is 
examined . 

2 . Personnel Management System 

The Officer Personnel Management System (OPMS) is a 
comprehensive plan encompassing the assignment, promotion, 
evaluation, etc. of Army officers. Career progression, and in 
particular education and training, is the aspect of OPMS which 
is of greatest concern. 
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With the cost of equipping the military rising at a 
dramatic rate, the pressure has been building to cut back in 
other areas of the budget. In 1976 the Senate Armed Services 
Committee recommended reductions in the Army training organ- 
ization. This was followed early the following year by the 
Office of Management and Budget's proposed minimums for 
education and training resource levels. These levels were 
far below what was then considered acceptable by the Army 
trainers. Since no major changes had been made in the 
system for many years (pre-Vietnam) the Chief of Staff of the 
Army directed in August 1977 that a Review of Education and 
Training of Officers (RETO) be conducted in order to: 

Determine officer training and education requirements 
based on Army missions and individual career development 
needs. Based on those requirements, develop training 
and education policies and programs which combine self- 
development, unit development, and institutional develop- 
ment in a phased schedule from precommissioning or 
preappointment training through career completion. 

[Review of Education and 
Training for Officers, 1978, 

p. I - 3] 

The recommendations of that study called for a 
complete revision in the manner in which the Army trains its 
officers. At present many of these recommendations have been 
approved for implementation while others have been deferred 
pending further review. Figure 5 depicts the difference 
between the current and proposed systems. 

Aware of the impending conversion to the new system. 
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1978 



the Leadership Action Planning Conference stressed the need 
to integrate plans for leader development with those RETO 
actions approved for execution. Accordingly, the plan for 
assessing the effectiveness of leadership training must be 
written in terms of the future state of training within the 
Army . 

The elements of the proposed system include: 

a. Accession Assessment Center 

Potential Army officers will be tested using 
assessment center technology on motivation and leadership 
potential, as well as the traditional medical and physical 
fitness exams, prior to acceptance into ROTC training. 

Military Academy candidates are currently exposed to this 
selection process. 

b. Basic Course 

All newly commissioned officers will attend an 
expanded basic course. Upon arrival at the school they will 
be required to pass a diagnostic test. This first step in a 
series of Military Qualification Standards (MQS) will insure 
a common body of knowledge and skills regardless of the 
commissioning source. 

Previously twelve weeks in length, the new 
courses will average nineteen weeks. The officer will be 
taught those skills required in the first three years of 
service. He/she will validate these skills through on-the-job 
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performance in order to become MQS II qualified. 

c. Temporary Duty Schooling 

Some of those skills which were previously 
taught in the Advanced Course may be covered in the expanded 
Basic Course or delayed until staff schooling. The majority, 
however, will be covered in temporary duty schools such as a 
Pre-Company Command Course of approximately five weeks dura- 
tion. . Throughout this stage of an individuals career fvinc- 
tional (technically oriented) schooling in a TDY status will 
be offered. Self-development by means of non-resident instruc- 
tion, college courses, and a professional readings require- 
meat will be strongly encouraged. Before the tenth year of 
service all officers must have completed MQS III to be 
eligible for staff schooling. 

3 

d. Combined Arms and Services Staff School (CAS ) 

3 

All new majors will attend CAS to provide them 
with some training in staff procedures. Approximately seven 
weeks in length, the proposed course of instruction calls for 
24 hours of leadership instruction (ethics, group d 5 mamics, 
and organizational effectiveness) . 

e. Command and General Staff College (C&GSC) 

3 

Approximately 40% of those majors completing CAS 
will be given advanced training in general staff skills at 
C6cGSC (or the equivalent sister service school) . Initially 
RETO recommended 20% attend this ten month school . In April 
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1980, however, the Commanding General, TRADOC, approved the 
elimination of the Advanced Courses providing the present 
40% attendance rate be retained. 

f. Precommand Course 

Lieutenant Colonel's and Colonel's selected for 
command now attend five seperate TDY courses (legal, mainten- 
ance, training, tactical, and command development) . It is 
proposed that these courses be tailored to the position. 

This tailoring would require a detailed front-end analysis of 
each command billet. 

g. Senior Service College 

Little change from the present system is called 
for in the new system. The conduct of joint and combined land 
warfare, as taught at the Army War College, remains a current 
training requirement. 

h. General Officer Development 

To provide a system to meet the continuing 
education and development needs of General Officers a three 
phase training program was proposed. It consists of trans- 
itional, orientational, and developmental training. Specif- 
ically called for are a two to six week interassignment 
course, periodic mandatory executive update seminars, and 
major command sponsored periodic Senior Commander /Manager 
workshops . 

The salient points of the new education and training 
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system, with regard to leadership training, are: (1) the 
use of assessment center technology/methodology is actively 
being pursued; (2) primary emphasis on leader development 
will take place in the units, schools being reserved for 
technical (hard skill) subjects, and; (3) with the expansion of 
the basic course fewer classes per year will be held at each 
school. With fewer classes the class size will have to 
increase to maintain the same throughput. This, in turn, 
will limit alternative methods of instruction necessitating 
increased use of the lecture. Redirection of assets from 
the Advanced Courses may obviate this problem. 

3 . US Army Leadership Training 

In 1971 the US Army War College was directed by the 
Chief of Staff, General William C. Westmoreland, to study 
Army leadership. Their findings, along with those of the 
Continental Army Command's Leadership Board, pointed out 
the wide disparity in satisfaction with Army leadership 
across ranks. This despite a common understanding of its 
importance. The recommendations made in this study. Leader- 
ship for the 1970 's , ushered in a decade of considerable 
research. 

It is from this study that the Army War College's 
Leadership Monograph Series began. The series is "dedicated 
to keeping Army leaders informed on a broad range of perti- 
nent techniques of leadership and management" [["Consolidated 
Army War College Leadership Monograph," 1975, p. iii[]. On 



80 



1 September 1974 ADMINCEN asstmed responsibility for the 
continuation of the series from the War College, 

In 1975 the Deputy Chief of Staff for Personnel 
tasked ADMINCEN to develop a single leadership model to 
provide for progressive leadership instruction, identify 
leader skills required at each level of experience, and iden- 
tify leadership training requirements from precommissioning 
to senior service college levels. Monograph # 7 of the 
series, "A Progressive Model for Leader Development,” rep- 
resented the first step toward that goal. 

The model developed by Clement and Zierdt Cl975]] is 
depicted in figure 6. It proposes that leader development 
is contingent upon the interaction of three variables: 
motivation (attitudes); skills and knowledge, and opportunity. 
Assiiming that soldiers generally enter the military system 
with a "positive attitude, and. .. reasonably well motivated” 
^Clement and Zierdt, 1975, p, 16]], development is primarily 
dependent on: (1) The skills and knowledge acquired in the 
service school system, and; (2) The opportunities in the unit 
which the personnel management system makes available. This 
second factor addresses the unit commander's responsibilities 
in developing his/her subordinates by providing the supportive 
climate necessary for the application of skills and knowledge. 

The development of a base line doctrine followed in 
the eight monograph of the series. Its objective was to 
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arrive at a "taxonomy" of managerial and leadership dimensions 
by position level. A taxonomy (a classification scheme that 
clarifies terms and concepts which would otherwise remain 
vague) was desired in order to permit the evaluation of the 
effectiveness of training programs by specifying, in explicit 
behavioral terms, what constitutes effective organizational 
leadership. Clement and Ayres (^19763 define organizational 
leadership as reflecting "two primary characteristics: on 
the one hand, leadership effects are oriented toward organ- 
izational objectives; on the other, leadership roles are 
established by the organizational structure so that positions 
of leadership are imposed on the group" {^Clement and Ayres, 
1976, p. 3]. 

The initial development of these dimensions was 
based on a review of the literature which included; Ohio 
State studies; University of Michigan studies; and other 
behavioral, communication, military, and management literature. 

From this review, nine dimensions of organizational 
leadership and five organizational levels were identified, 

[see figure 7]. Appendix A defines each dimension and sub- 
dimension. 

In May 1977 Monograph # 9, "Organizational Leadership 
Tasks for Army Leadership Training," applied the first phase 
of TRADOC's instructional systems design (ISD) model to 
those dimensions previously identified. The first step was 
to catagorize the dimensions by skill area, this resulted in 
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FIGURE 7. A MODEL FOR ANALYZING 

ORGANIZATIONAL LEADERSHIP 
SOURCE: LEADERSHIP MONOGRAPH NO. 8 
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DIMENSIONS," Stephen 0. Clement ft Donna B. Ayres, 1976, p. 14. 
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"Intrapersonal" was added to the "Communication” dimension. 

A number of products have been designed to support 
the development of leadership training at the service schools 
based on the matrix. USAIS has published a leadership task 
list, more comprehensive than Monograph # 9, which was the 
result of the second leadership seminar held in March 1979. 

This list is intended to provide a common base for course 
development. It was, however, acknowledged that how and 
when the material is integrated into the present courses 
(if at all) is still the prerogative of the individual schools. 
USAIS has also developed lesson plans, complete with tasks, 
conditions, and standards, for each dimension and level up 
through company grade officer. These are available to the 
schools to assist them in their leadership training program. 

Despite these efforts to standardize leadership 
training, the schools have been slow to adopt the material. 

Some training developers feel resistance from the ISD managers 
over the difficulty of expressing the dimensions in the 
proper format. Clement and Ayres addressed this 

problem in Monograph # 9 : 

Up to now, most of our training focus has been grounded 
in the "transfer of identical elements" in which learning 
is presumed to occur only if the identical elements 
present in the old situation are present in the new sit- 
uation. This learning theory is fine if we can success- 
fully identify all of the elements to be learned. 
Unfortuantely , "soft skills" such as leadership skills 
are not amenable to a comprehensive delineation of dis- 
crete elements. For this reason, a shift to the trans- 
fer-of-principles theory allows us to assume that the 
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leader can generalize his learning; the learner need 
not be aware of the presence of identical elements in 
a stimulus situation because transfer will occur when 
he applies those principles having sufficient generality 
to cover a class of stimuli which are learned in past 
specific situations. 

[^Clement and Ayres, 1977, p. 3] 
Field Manual 22-100 has been rewritten to support 
leader development at the individual and unit level through 
the adoption of the organizational leadership matrix. This 
manual, the first of three, is designed for the lower and 
first- line levels of the matrix. Plans currently call for 
two additional manuals, one for middle and top (field grade) 
officers, and eventually a senior (executive) level manual. 

In a separate effort to support the units, a training 
package, called Leadership and Management Development Course, 
(L&MDC) , has been implemented. Initially fielded in 1975 at 
three installlations , its success warranted expansion to all 
installations and division level units. 

An experience based workshop, it is conducted by 
instructors certified by the Organizational Effectiveness 
Center and School. The purpose of the course is to: (1) 
Develop an understanding of one theory of work-team develop- 
ment; (2) Develop an understanding of the situational 
appropriateness of various managerial styles, and; (3) Intro- 
duce communication skills essential to effective mission 
accomplishment through people, which will make participants 
more effective as group managers and members. 
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The course is a one week workshop presented to small 
groups (10 - 12) composed of peers (equivalent rank) who do 
not have a work relationship. A one day follow-up is con- 
ducted approximately six weeks after the course completion. 

It was originally designed for E-5 through 0-3 and equivalent 
civilian grades but since then a modified version for 0-4 
through 0-8 has been designed. 

Appendix B lists the key learning objectives of 
the L&MDC in order of presentation. This list should provide 
sufficient insight into the course curriculum. The conduct 
of the course utilizes the methods of adult or experiential 
education. "They will find themselves actually involved in 
problem solving situations which provide real feelings of 
satisfaction or frustration. They will be providing and 
receiving information about their leadership behavior in the 
course and that of their co-participants” OEC&S, 1979, 

p. 23. 

A major reason for the program's success is that 
it is unit-based training, that is, personnel discuss real- 
life problems and situations during the workshop and return 
to their units immediately after training. However, it has 
been realized that because the participants do not have a 
work relationship, there is no opportunity for mutual rein- 
forcement of learning upon return to the unit. 

Evaluation of L&MDC currently consists of administering 
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a survey at the end of the course and six weeks later at the 
follow-up session. The participant survey is designed to 
solicit feedback from the students on both the presentation 
and the perceived relevancy of the course. 

In a 1975 study, A Comparison of Three Experimental 
Learning Programs: SKIM, PET, L6eMD , the advantages and dis- 
advantages of L&MDC were discussed. These remain applicable 
today and are presented in Appendix C. 

4. Summary 

In this section the three major subsystems of leader 
development within the US Army have been examined, the organ- 
ization for training development, the officer personnel man- 
agement system (training and education subsystem) , and the 
leadership training model. Throughout, it has been demon- 
strated that this is a system subject to many changes, yet 
one which has recently been given new guidance and direction. 
Much of this direction, in the form of influencing the RETO 
study, came from the other military services. In the following 
sections the leader development of these services are 
presented. 

B. UNITED STATES NAVY 

1. Organization for Training 

Unlike the Army, the Navy operates a far more cen- 
tralized education and training system with respect to leader- 
ship training. At the top of this system is the Deputy Chief 
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of Naval Operations for Manpower, Personnel and Training 
(OP-01) . Directly subordinate is the Human Resources Manage- 
ment Division (OP- 15) whose mission is to: 

Develop plans and policies and establish programs, pro- 
cedures, and improved practices in the Naval organization 
which support increased unit and personnel effectiveness, 
improved leadership and management, enhanced career sat- 
isfaction and increased personnel motivation. 

USN, Officer Personnel Newsletter 
Fall 1979, p. 3 

To fulfill this mission a Human Resource Management 
Support System (HRMSS) has been planned to coordinate the 
Navy's programs in the following five areas: (1) Organiza- 
tional Development; (2) Leadership and Management Education 
and Training; (3) Equal Opportunity; (4) Overseas Duty 
Support, and; (5) Drug and Alcohol Abuse. This section will 
concern itself solely with the leadership program. 

As training policy coordinator, the Director, HRM 
Division (OP-15), works closely with the Chief of Naval 
Education and Training (CNET) on the development, management, 
implementation, and evaluation of leadership training, 
specifically the Leadership and Management Education and 
Training (LMET) courses. Currently this relationship between 
OP- 15 (program sponsor) and CNET is modified so that OP- 15 
retains curricultim control over the LMET program, t 3 q)ically 
a CNET responsibility. 

In the area of research there are two organizations 
which work on issues related to leadership training. The 



Office of Naval Research has primary responsibility for the 
theoretical or basic research which is conducted. This 
research- in- science phase is commonly referred to as 6.1 
funds and involves the awarding of contracts to scholars in 
the behavioral science field (among others) . 

Progressing along the continuum of research develop- 
ment to the 6.2 funds the major effort in this research- in- 
technology phase is aimed at solving military problems, short 
of major program development. That is, how can the efforts 
in the first phase now be applied to problems in the military. 
This job of applied research is undertaken by the Naval Per- 
sonnel Research and Development Center, (NPRDC) , San Diego, 
whose mission is to: 

Conduct human resources RDT&E in the areas of manpower, 
personnel, education, and training, and serve as the 
coordinating agency for all human resources RDT&E for 
the Navy. 

[DN, NPRDC, 1978, p. iii] 

2 . Personnel Management System 

A Naval officer's transition through the training 
and education system depends to a great extent on his or 
her warfare specialty. There are three major divisions of 
officers within the Navy: Unrestricted Line, Restricted 
Line, and Staff Corps. This discussion will focus on the 
largest division, the Unrestricted Line, and specifically 
on its major specialty. Surface Warfare. Other warfare 
specialties in this division include Aviation and Nuclear 
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Submarine . 



Officers entering the Surface Warfare Community 
are sent to a 16-week Surface Warfare Officers School (SWOS) 
where emphasis is placed on those functional areas necessary 
to prepare the individual for watch and division officer 
duties. Prior to the introduction of LMET at all initial 
training sites the Surface Community was the only specialty 
offering its newly commissioned officers leadership and manage- 
ment training. QSpencer, 1978, p. 121] 

After SWOS the initial sea tour will usually be as a 
division officer in either the Engineering, Weapons, or 
Operations Department. Within this tour he/she is expected 
to qualify as a Surface Warfare Officer. Qualification is 
guided by the Personnel Qualification Standards (PQS) program. 
PQS is an outline of the various items an individual must 
accomplish to be certified (certification being made by the 
ship's Captain). It consists of: (1) Fundamentals, knowledge 
necessary to do the job; (2) Systems, man/ equipment inter- 
face; and, (3) Watch Stations, procedures. Qualification is 
a prerequisite for attendance at the next level of formal 
training. Surface Warfare Officer Department Head Course. 

Attendance at this course during the first shore tour 
qualifies the officer to serve in any one of the principal 
department head positions. During the next sea tour the 
officer is encouraged to prepare himself/herself for the 
command qualification examination which is required for 
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command of a surface ship. After three years (18 months each 
on two classes of ships) the officer usually commences the 
second shore assignment. 

During this assignment attendance at the Postgraduate 
School or staff college can be anticipated (if qualified) . 

The one-year College of Naval Command and Staff (equivalent 
to the Army's Command and General Staff College) is collocated 
with the Navy's College of Naval Warfare. Officers do not 
attend both schools. "The Navy places little emphasis on 
attendance at a staff or war college -- just over 10 percent 
of the eligible Naval officers attend at [either] of these 
levels. Command, not formal education is a pathway to stars" 
[Review of Education and Training for Officers, 1978, p. G-2- 
11]. Training at both colleges is divided into: (1) Maritime 
strategy and policy and their interrelationship with foreign 
policy; (2) Defense economics and decision making to allocate 
national resources in accordance with national goals and 
strategy, and; (3) The emplo 3 nnent of naval forces to accomplish 
the four-fold mission of strategic deterence, naval presence, 
sea control, and projection of power overseas. 

Previous specialized training (e.g. Postgraduate 
School during the first shore tour) will be utilized the 
first two years of the second shore assignment with the 
possibility of staff college during the last year. 

After selection for lieutenant commander (04) and 
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recommendation for assignment as an executive officer (or 
command in a very few cases) the officer attends the Pro- 
spective Executive Officer (PXO) Course. This five-week 
course (two weeks of leadership training) had previously been 
the first formal leadership and management training received 
regardless of specialty. 

During the third and subsequent shore assignments the 
officer can expect to fill any of the following five categories 
of billets: (1) Operational assignment requiring utilization 
of Surface Warfare Officer qualifications; (2) Billets in the 
area of his/her subspecialty; (3) General unrestricted line 
billets appropriate to the grade; (4) Senior service college 
attendance, or; (5) Washington duty. 

The majority of the Surface Warfare Officer commander 
(05) sea assignments are as Commanding Officers of ships. 
Initially screened for command the year in which they are 
selected for promotion, the individual's record is considered 
each year until selected for command or until he/ she enters 
the secondary promotion zone to captain (06) . Those selected 
are ordered to command via the six- week Prospective Commanding 
Officer (PCO) Course which include two weeks of LMET. 

In sxammarizing the Navy's training and education 
system, it is apparent that it has had a significant impact 
on the remodeled Army system. The attainment of Military 
Qualification Standards during the Army Officer's first ten 
years is very similar to the Navy's Personnel Qualification — 
Standards . Both programs being designed to insure that 
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officers obtain the fundamental skills essential to their 
specialty. The assignments and schools addressed in this 
section are summarized on figure 9. 

3 . US Navy Leadership Training 

The 1975 BUPER's study refered to in the previous 
section was a turning point in the Navy's approach to leader- 
ship training. The program which has evolved from that study 
is clearly the most integrated and systematic program of any 
service. Its development warrants a detailed examination. 

Lyle M. Spencer ^19783 , in summarizing the results of 
that study, noted the following; 

a. A majority of naval commissioned and non-commis- 
sioned officers did not receive any leadership and management 
training at key ascension points in their careers (e.g. Div- 
ision Officer, Department Head, Executive Officer, etc.). 

b. 157 different leadership courses existed at that 
time [1975] in the various commands . 

c. An examination of the content of the existing 
courses found: (1) There was no standard curricula or 
consensus about what knowledge or skills were needed to 
perform effectively as a Navy leader. Courses consisted of a 
combination of Navy tradition, rules, regulations, civilian 
academic management theories , and a few offerings from the 
behavioral sciences; (2) Courses lacked any foundation in 
empirical research; (3) Courses were 90 percent cognitive and 
10 percent experiential. They consisted primarily of talking 
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at people about leadership and management in formal classroom 
lecture formats rather than teaching them actual leadership 
and management skills, and; (4) While the courses were well 
received, the participants had difficulty in relating what 
they had learned to their jobs. 

To correct these problems BUPER's (now OP-01) sought 
to develop an empirically based training program, that is, 
based on the actual skills used by superior performers. 
Confronted with the task of identifying those competencies 
(skills, knowledge, and motivation) which determine or pre- 
dict effectiveness in a leader a research method developed by 
'David McClelland [1976] was used. 

Realizing that it is much easier to reach consensus 
on who is an effective leader vice what makes an effective 
leader, the first step was to identify a group of superior and 
a group of average leaders for each ascension point. These 
ratings were made by the commanding officers and, where pos- 
sible, verified by examination of retention rates, "E” awards, 
and inspection scores. This was first undertaken in San 
Diego during the September -- October 1976 time frame. A sample 
of eighty- two officers and enlisted personnel were obtained, 
however, only 51 ratings were received on the 82 people inter- 
viewed. Of these 51, 30 were rated superior and 21 average. 

The next step involved interviews utilizing a tech- 
nique termed Behavioral Event Interviewing (BEI) . In it. 
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the leader is asked to relate some critical incidents, that 
is, important success and failure experiences they have had 
in their present jobs. Specifically requested was information 
concerning; (1) What led up to the experience; (2) Who was 
involved; (3) What they (the leader) felt, wanted, or intended 
in responding to the situation; (4) What they actually did, 
and; (5) The outcome or results of the action taken. After 
several incidents from each person were gathered, a comparison 
of those factors which distinguished between the superior and 
average leaders was made. 

Two or more researchers independently examined the interview 
transcripts ... to identify distinguishabe individual char- 
acteristics which appeared to differentiate superior from 
average performers and those which were expressed with equal 
frequency by all performers were identified. 

[Klemp, et al., 1977, p. 12] 
From the initial interviews of the rated personnel in 
the Pacific Fleet twenty-seven competencies, grouped into 
five clusters, were identified as distinguishing leaders. 

These competencies are listed in the left hand column of 
figure 10 as "LMET Competencies." 

In order to validate these competencies the same 
procedures were applied to a similar cross-section of person- 
nel from the Atlantic Fleet in February 1977. This time, 
however, the leaders were interviewed prior to being rated 
by their superior. Based on a factor analysis of the Pacific 
Fleet results (the fifth factor. Coercion was not used) a 
prediction of group membership (superior/aveirage) was made 
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for the Atlantic Fleet. A discriminant function showed that 
in 67.9% of the cases the leader was correctly identified as 
superior or average (p < .001). For this analysis only 78 of 
the 100 leaders interviewed were rated (38 superior and 40 
average) . [Klemp, et al. , 1977] 

In May 1977 a committee comprised of the representa- 
tives of the program sponsor (OP-15), CNET, and the Chief of 
Naval Technical Training met to design the first four of 
fourteen ascension level specific courses. To the right of 
the initial competencies in figure 10 is a list depicting 
how the competencies were formed into curriculum clusters. 

During the initial stage of Curriculum development, 
"management control" .. .was divided into two areas 
"problem solving" and "process management." The reason 
for doing this was that students could first look at 
problem solving through planning, processes, concept- 
ualizing, and forming an action plan. Process Manage- 
ment then deals with optimizing people and resources. 
Coercion, being a type of influence, was established 
that even though it separated superior and average 
performance, superiors used it less and as a last 
resort. Therefore, for instructional purposes, it was 
included under the dimension of "skillful use of ' 

influence." Finally, the cluster title "task achieve- \ 
ment" was changed to "concern for efficiency and ' 

effectiveness ." 




QMangus , 1980, p. 28] 

A trial course was conducted in June 1977 with 
favorable results. During this period of curriculum devel- 
opment and pilot testing a second validation effort was 
undertaken . 

This second effort (August through October 1977) 
sought to gather data from a sufficiently large enough sample 
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to verify competencies across all levels and warfare com- 
munities. To do this a battery of nine paper and pencil 
tests were developed covering fifty separate scales or 
variables. This test was administered to 1,200 officers and 
enlisted personnel. 

A competency was regarded as validated if one or 

2 

more of its indicator variables was significantly correlated 
with the superior's rating of performance. This in one or 
more of the following billet groups: (1) Commanding Officer 
and Executive Officer; (2) Department Head and Division 
Officer; (3) Master and Leading Chief Petty Officer, and; 

(4) Leading Petty Officer and Petty Officer. 

Sixteen of the original twenty- seven competencies 
from the Pacific and Atlantic Fleet interviews were validated 
in this manner. These competencies, grouped by curriculum 
cluster, are presented in figure 12 and defined in Appendix D. 

These sixteen competencies form the core of the two- 
week LMET course with variations on course content being made 
for each ascension level school. LMET schools have been 
established for the following levels; PCO/PXO, Department 
Head, Division Officer, Chief Petty Officer, and Leading 
Petty Officer. Recruits will not receive LMET training but 
will be exposed to the concepts as modeled by their instructors 
and company officers. The instructors at these LMET schools 
are all trained and certified at the Human Resources Management 
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School, Naval Air Station Memphis, in the twelve-week LMET 
Instructor Course. 

During each course the student progresses through a 
five-step process; 

a. Recognition. Each competency is explained in 
terms of desired knowledge, skills, attitudes, etc. and 
how this contributes to the performance of the superior Navy 
leader . 

b. Understanding. The participant translates the 
competency by means of his/her own experiences into language 
with which they are comfortable. 

c. Self-Assessment in relation to the competency. 

Each student is able to identify areas in which he/ she is 
strong or needs new skills. 

d. Skill acquisition and practice. All skills are 
practiced with emphasis on those the individual has identified 
as needing improvement in. 

e. Job application. Participants develop individual 
action plans for application on their job. This step rein- 
forces learning and gains commitment to the competencies 
desired by the Navy. 

At the unit level, the teaching of formal LMET courses 
by uncertified instructors is prohibited, as is the modifica- 
tion of LMT courses (those addressed in the 1975 study) to 
include LMET precepts and procedures. It is envisioned 
that LMET team building workshops can be designed which will 
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be administered through the Human Resources Management Support 
System. These workshops will bring to the mit level the 
latest refinements of the LMET program and reinforce individual 
skills. 

It is important to note that the changes in course 
content were based on validation and not evaluation studies. 
Presently a funding shortage is prohibiting a comprehensive 
external evaluation (measures designed to assess actual 
changes in job behavior or organizational performance) as 
designed by NPRDC. On the other hand, CNET is progressing 
toward an internal evaluation plan to determine whether the 
course is actually teaching what it was designed to teach. 

This effort centers on: 

a. The development of a standardized student critique 
form for all LMET sites. 

b. Posttraining tests to assess student performance. 
McBer and Company is still involved in this aspect of the 
evaluation. 

c. Assessment of the student by the instructor. 

It is estimated that one year will be required for 

funding and program writing. All inputs to the evaluation 
will be computer scored and analyzed for possible course 
revisions . 

One matter which complicates the evaluation process 
is the Navy policy that personnel will not be denied LMET 
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training for the purpose of establishing a control group. 

In a proposed Human Resources Management Support 
System (HRMSS) Navy Training Plan the establishment of an 
Evaluation and Information office is called for. The 
purpose of this office will be to formulate a simimative 
evaluation design and collect data on all aspects of the 
HRMSS. This will be an ongoing effort and include LMET as an 
element of that system. 

The principal benefit in reviewing the Navy's 
approach to education and training has been in contrasting 
the highly centralized approach towards leadership training S 
with the Army's decentralized system. The result being a 
fairly rapid implementation of a major program (LMET) as 
compared to the many years spent in developing a base line 
doctrine for leadership training in the Army. In the next 
section another centralized approach. is reviewed, the 
US Air Force. 

C. UNITED STATES AIR FORCE 

1 . Organization for Training 

Like the Navy, the Air Force operates a very cen- 
tralized training system with regard to its Professional 
Military Education (PME) , The Commander, Air Training 
Command, has proponency for each level of education in an 
officers career from precommissioning through senior service 
college. Under his operational control fall the following 
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PME schools Squadron Officer School (SOS) , Air Connnand and 
Staff College (ACSC) , and the Air War College (AWC) . All 
of these schools are located at Maxwell Air Force Base, 
Alabama . 

These schools differ from the purely technically 
oriented schools in that PME encompasses the knowledge 
unique to the profession of arms in general and the employ- 
ment of aerospace power in particular. Within the schools’ 
curricula are; national security affairs, military history, 
leadership, management, professional ethics, and standards 
of behavior. 

Through this organizational differentiation between 
PME schools and the functional or technical schooling the 
Air Force, more than any other service has developed the 
distinction between education and training. Education being 
the more general and abstract while training more specific 
and concerned with the development of job specific skills. 

Each of the PME schools listed above has its own 
leadership curriculinn development branch. The doctrine 
and policy which guides the programs is formulated at the 
Air Force Chief of Staff level by the Director of Personnel 
Plans . 

Research in leadership related areas is conducted 
by the Occupational Measurement Center at Lackland AFB. 

This organization performs a task analysis for officer 
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positions and recommends the type of training or education 
needed to perform that task. Their work has been used to 
identify the eight dimensions of leadership which are 
detailed in the third section. 

The Air Force Human Resources Laboratory, Brooks AFB, 
has provided limited research in the leadership area, most of 
which has been confined to literature reviews. Both the 
laboratory and the center are under the operational control 
of the Air Force Systems Command. 

A third agency which has indirectly aided the 
research effort is the Leadership and Management Develop- 
ment Center (LMDC) , Maxwell AFB. This agency is tasked with 
identifying systematic issues in leadership and management 
for Air Force commands. Their mission is similar. to the 
Navy's Hxaman Resource Management Center ' s /Detachment ' s and 
the Army's Organizational Effectiveness Staff Officers. 

LMDC teams administer surveys designed to measure 
such variables as satisfaction, productivity, attitude, and 
leadership traits and behaviors at all levels of the base's 
or xmit's chain of command. Over the past two years the center 
has compiled sufficient data to conduct an analysis of the 
leadership variable. These data have been provided to the 
curriculum developers at each PME school to aid them in the 
design of their leadership training programs . 

2 . Personnel Management System 

The career pattern, with respect to formal schooling. 
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for the Air Force officer is different than that of either 
the Army or Navy system reviewed so far. Officers other 
than rated officers (pilots and navigators) and certain 
technical specialties (missle, communications, computer, 
aircraft maintenance, and security police) proceed directly 
from their commissioning source to an assignment. Those that 
do go to a school after commissioning receive technical 
instruction only, no leadership or management training. 

The Squadron Officer School provides the first level 
of formal Professional Military Education to all officers 
between their fourth and eight year of service. The SOS is 
a short (11 week, temporary duty) course which focuses on 
command and staff capabilities rather than further specialty 
development. It is essentially a leadership school for 
company grade officers. 

The Air Command and Staff College provides the inter- 
mediate level of PME to approximately 20 percent of Air Force 
officers who attain the rank of major. A 40 week resident 
course, its mission is to "provide mid-career officers with 
the skills, knowledge, and understanding that will enhance 
their value to the Air Force for the balance of their careers 
in responsible command and staff positions" [[Review of Educa- 
tion and Training for Officers, 1978, p. G-l-bJ. This mission 
has been translated into eight specific course goals in the 
areas of; common staff goals, specific staff skills, specialist 
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skills, broaden knowledge of the Air Force, broaden view 
beyond the Air Force, and research. 

At the senior level the Air War College provides PME 
to approximately eight percent of those officers selected 
for the grade of lieutenant colonel. The College's mission 
is to prepare selected officers for key command and staff 
assignments which it does by placing emphasis on airpower 
strategy and emplo 5 mient. 

3 . US Air Force Leadership Training 

The preponderance of an Air Force officer's leader- 
ship training occurs at the Squadron Officer School. It is 
therefore appropriate to detail that course's curriculum as 
an explanation of their program. 

The course is conducted four times annually with a 
class ranging in size from 550 to 650 lieutenants and cap- 
tains. Each class is further subdivided into sections of 
twelve on the basis of a list of thirty variables (e.g. 
duty, command, age, sex, etc.) to insure a thoroughly diver- 
sified group. Each section is provided with a section 
commander who is a member of the faculty. 

During the first two weeks the students receive 
limited instruction on leadership theories and take part in 
many group problem solving exercises (e.g. Project X). The 
section commander utilizes this assessment phase to profile 
each officer along a series of leadership dimensions. These 
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dimensions are; problem solving, organizing skills, inter- 
personal skills, forcefulness, willingness to lead, flexi- 
bility, motivation, and acceptance of responsibility. 

At the begining of the third week the section 
commander, and an attached member of the staff who has been 
assisting the commander, feedback to the students their 
perceptions and recommendations for improvement. A program 
is designed with the officer to develop either the areas 
discussed as needing work during the feedback or areas ident- 
ified by the officer himself /herself. Plans are being made 
to include self-assessment techniques for students to aid 
them in developing their programs. 

Much of the remaining training is experientially 
based. The student continues to work within the group on 
structured exercises and in competitive sports, concentrating 
on the dimensions specified in the plan. For example, an 
individual who needs/wants improvement in organizing skills 
would be made the coach of the section baseball team for a 
week, responsible for organizing practices, making team assign- 
ments, etc. 

The assessment of the program's effectiveness has been 
primarily limited to internal evaluations. Students complete 
questionnaires at course completion and six months and one 
year after graduation. Approximately 50 percent of the 
replies have been favorable with the most negative comments 
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concerning the relevancy of sports to the officer's assign- 
ment back at his/her duty station. The individual's superior 
also receives a questionnaire about the officer's performance 
after attendance as compared to their pre-attendance perform- 
ance. In general, their comments are more positive than the 
students . 

The concepts behind this program were based on the 
assessment center process developed by the Office of Strat- 
egic Services (OSS) in the 1940 's to select agents and the 
American Telephone and Telegraph Company's Management Pro- 
gress Study of the 1950 's. Since that time many organizations 
have utilized this methodology to- select or, as the Air Force 
has done, to develop its leaders and managers. A closer 
look at assessment centers is taken in a later chapter. 

At the other two PME schools the subject of leadership 
is not addressed in as formal a manner as it is at SOS, nor 
is there a service-wide leadership training program at the 
unit level such as the Army's L<ScMDC. 

D. UNITED STATES MARINE CORPS 

1 . Organization for Training 

The Marine Corps operates the most centralized 
agency for leadership instruction of any service. The Leader- 
ship Instruction Department (LID) at Marine Corps Development 
and Education Command (MCDEC) , Quantico, Virginia, is respon- 
sible for: (1) Conducting leadership training in the formal 
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schools; (2) Conducting special leadership training and brief- 
ings; (3) Developing, testing, and revising leadership courses 
and manuals, and; (4) Studying prevailing conditions, attitudes 
prejudices, and behavior affecting leadership training. 

[USMC Order 5390.2B, p. 4] 

All formal schools (The Basic School, Amphibious 
Warfare School, Communication Officers School, and Command 
and Staff College) are located at Quantico, LID, as a 
single agency, provides a cadre of instructors who teach 
leadership at each level. Currently LID is the only agency 
in the Marine Corps which conducts research in the area of 
leadership . 

Another major aspect of the Marine Corps leadership 
program is its unit leadership training (details of which 
are addressed in the third section) . The Equal Opportunity 
Branch is responsible to the Director, Manpower, Plans, and 
Policy Division for the development, implementation, and 
monitoring of this program. Unit commanders are responsible 
for local implementation of the program to include preparation 
of an Annual Leadership Training Master Plan. 

2 . Personnel Management System 

The influence of the Marine Corps on the proposed 
Army education and training system is felt primarily in the 
precommissioning accession source. Almost 50 percent of 
the Marine officers are commissioned through the Platoon 
Leaders Class (PLC) program which requires attendance at 
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two six-week summer camps (one after enrollment and one 
inmediately prior to college graduation) or a ten-week camp 
if enrolled in the junior year of college. These camps are 
operated by the Officer Candidate School. Other sources of 
commissioning include the Naval Reserve Officer Training 
Corps, the Naval Academy, and Officer Candidate School. 

Initial entry training for Marine officers is unlike 
any other service. All new officers attend The Basic School 
(TBS) with specialty designations being made as a result of 
performance during the course. Functional training follows 
The Basic School, if required, according to occupational 
specialty assigned. "For a significant number of Marines 
(including some who will be advanced to the field grade) 
this will be their only extended formal resident schooling” 
[Review of Education and Training for Officers, 1978, p. G-2]. 

At the intermediate level only about 30 percent of 
the eligible officers attend resident instruction. The 
Amphibious Warfare Course (AWC) is the principal intermediate 
level professional school in the Marine Corps. It emphasizes 
Marine air-ground task forces in amphibious operations to 
prepare captains for command or staff assignments in battalion 
and regimental size forces of the Fleet Marine Force. Other 
schools available include the Communications Officers School 
and a number of Army Officer Advanced Courses. The impact of 
the Army decision to phase out its intermediate level courses 
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and replace them with temporary duty schools is yet to be 
determined. 

The Marine Corps Command and Staff College stresses 
intellectual processes, oral and written communication, and 
the analysis, s 5 mthesis, and evaluation of problems to prepare 
graduates for high-level assignments. For top level education 
the Marine Corps utilizes other services' senior colleges. 

3 . US Marine Corps Leadership Training 

As promulgated in the Marine Corps Manual the objective 

of Marine Corps leadership training is to: 

Develop the leadership qualities of all Marines to enable 
them to assume progressively greater responsibilities 
to the Marine Corps and society. All Marines, officer and 
enlisted, will participate in leadership training designed 
to achieve this objective. 

[Marine Corps Order 5390. 2B, 
1979, p. IJ 

The leadership training refered to is a composite of 
what other services refer to as equal opportunity, human 
relations, racial awareness, etc., as well as those 
topics which might be considered more traditional leadership 
subjects. Leadership is defined in the Marine Corps Manual 
[paragraph 5390] as "the sum of those qualities of intellect, 
human understanding, and moral character that enables a 
person to inspire and to control a group of people success- 
fully." A key distinction in this definition vis-a-vis the 
other services, is the use of the word "control" as opposed 
to influence. The rational behind this is that "control" 
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implies a more authoritarian approach than does the term 
"influence" which is consistent with Marine Corps philosophy. 

In The Basic School the orientation is towards pro- 
viding the newly commissioned officer with an understanding 
of the concepts of Marine Corps leadership and to prepare 
him/her as a discussion group leader for unit level training. 
The foundation of the concepts phase lies in the teaching of 
the eleven principles of leadership with opportunities pro- 
vided throughout the course to practice their application. 
These principles are listed in figure 12. 

The unit leadership training is designed to provide 
the Marine with instruction/ informat ion on topics such as 
alcohol and drug abuse, sexism, racial equality, etc. The 
characteristics of this program are set forth in Appendix 
E. The reason for subsuming these topics under leadership is 
that it is ultimately the leader who must deal with issues of 
this nature. Each commander is responsible for determining 
the goals and objectives for his/her unit based on the sit- 
uation and to design an Annual Leadership Master Training Plan 
to meet these needs. 

In order to prepare the company commander for deter- 
mining goals and objectives and in planning, organizing, con- 
ducting, and evaluating leadership training the next level of 
formal training, the Amphibious Warfare or Advanced Communica- 
tions Officer courses, is oriented towards this objective in 
its leadership training. 
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At the next level of schooling, Command and Staff 

College, leadership subcourses are designed to provide officers 

with a "command perspective" of leadership training. The 

table of contents of the College's handout on leadership 

provides the following interpretation of this orientation in 

its listing of subcourses: 

The Responsibility of Leadership 

The Military Profession 

Ethics and the Military Profession 

Developing an Ethical Code 

Moral Decision Making 

Substance Abuse 

Philosophy of Command 

[USMC, Command and Staff College, 
1979, p. i] 

Currently all efforts at assessing the effectiveness 
of the resident instruction center on internal evaluations, 
e.g. student critiques of the material presented. No formal 
external evaluation has been conducted, nor is planned for the 
near future. 

E. UNITED STATES COAST GUARD 

1 . Coast Guard Leadership and Management School 

As the smallest of the uniformed services, the US Coast 
Guard offers a completely different perspective on leadership 
training. Unlike the systems studied thusfar, there is no 
formal career progression for the USCG officer. There are no 
mandatory schools and/or assignments for the individual who 
aspires to flag rank. The organization for training which so 
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significantly impacted on the leadership training in the other 
services does not play as major a role here. 

In November 1975 the Coast Guard began development 
of a Leadership and Management School (CGLAMS) . Instituted 
in 1977 the school now conducts three courses at both its 
Yorktown, Virginia and Petalxjma, California training sites. 

(see Appendix F, CGLAMS Mission, Goals, and Objectives) In 
contrast to the other leadership schools, attendance at the 
two-week junior officer (03 and below) and the senior petty 
officer courses, as well as the one-week senior officer course 
is voluntary. District headquarters receive quotas for each 
school and select the attendees from among those individuals 
whose applications have been received. 

The purpose of the school is to develop the leader, 
to draw on his/her past experiences to increase their ability 
to analyze a situation and thereby select an appropriate 
style of leadership. In order to accomplish this the school 
requires that only those individuals who have demonstrated 
above average leadership competency be selected for attendance. 
The school is not designed to handle poor leaders sent for 
training by the commands. The underlying concepts of the 
resident courses are based on three basic principles: 

1. Leadership is a relation amoung several variables. 

2. Leadership is situational. 

3. Leadership requires flexibility on the part of the 

leader . 

[Marcott, 1978, p. 13] 



117 



The framework upon which the courses are built is an 
adaptation of Rensis Likert's concept of the relationship 
between three classes of variables: (1) Causal variables or 
stimuli such as regulations, policies, and leadership styles; 
(2) Intervening variables which act upon the individual or 
group, e.g. need orientation, motivation, experience, goal 
setting ability, personality, and a sense of responsibility, 
and; (3) End results, e.g. short and long term productivity 
and impact on htiman resources. The school contends that 
"the only thing we have control over, short of changing policy 
directives, is our leadership style (^and that] if we can 
increase our knowledge of and understanding of the intervening 
variables, perhaps we have a better chance of selecting an 
effective leadership style" [Marcott, 1978, p. 14]. 

To develop this understanding the courses involve the 
students in the study of groups, individuals, and situational 
leadership. In studying the group the students "are exposed 
to the difference between 'content' and 'process' and are 
invited to note how failure to observe process in a work 
group can inhibit task completion. .. they are aided in recog- 
nizing group norms and their importance ... the effect of 
cohesiveness within a group is discussed in depth, taking into 
account the impact that status has on group decisions" 

[liarcott, 1978, p. 14]. Of particular concern is the potential 
conflict between jimior officers, petty officers, and the work 
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group. The leader is given skills and practice in addressing 
this issue. 

Herzberg's motivation - hygiene theory, transactional 
analysis, and interpersonal communications (barriers, active 
and passive listening, non-verbal behavior, and feedback) 
are all covered in studying the individual. Here, as in all 
phases of the school, the principal teaching techniques are 
role playing, workshops, team teaching, etc. The lecture is 
seldom used as the course attempts to capitalize on the 
experiences of the participants. 

Mersey and Blanchard's Situational Leadership model, 
as presented in the previous chapter, is used to draw to- 
gether the knowledge of the individual and group in selecting 
a leadership style. "They learn to use their new found 
knowledge of the intervening variables to analyze their 
followers' maturity level. This knowledge gives them a 
better feel for what might be an appropriate or inappropriate 
'relationship behavior' in a given circumstance" [Marcott, 1978, 
p. 17J. 

Also covered is a modified version of the critical 
path method of modeling the work flow. This is presented to 
aid the leader in planning, scheduling, and controlling 
proj ects . 

In the two-week course this material is followed by 
a week of case studies drawn from actual Coast Guard experiences 
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geared to the individual's rank. The course concludes with 
a viewing of the film "12 O'clock High." The movie is 
shown in segments with an analysis and group discussion of 
the intervening variables, leadership styles, and the possible 
consequence of the two at the end of each segment . 

To standardize the principles of the courses 
within the Coast Guard, members of the school's staff have 
taught the leadership instructors at the Coast Guard Academy 
and the Officer Candidate School (the only two sources of 
commissioning) . The only difference in the courses is in the 
orientation of the case studies. 

The CGLAMS has been moving into other areas recently, 
most notably in the field of organizational development 
"after re-discovering that training alone, without the 
necessary change in the structure of the people subsystem, 
does not impact the organization" [Wehrenburg, 1980, Interview]. 
In the training area the school is researching the possible 
use of behavior modeling as a method of teaching. This 
approach consists of the following four major learning 
activities ; 

1. Modeling, in which trainees watch films of model 
persons behaving effectively in a problem situation. 

2. Role Playing, when the trainees practice and 

rehearse the effective behaviors demonstrated by the 
models. 1 

3. Social Reinforcement provided by trainer and trainees \ 

in the form of praise and constructive feedback. \ 
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4. Transfer of training is encouraged, to enable the 
classroom behavior to be effectively utilized on the 
job. 

[Kraut, 1976, p. 325] 

2 . Evaluation 

At Yorktown the Development Staff of the CGLAMS has 
been conducting an extensive evaluation of the leadership 
courses. A four phase approach, the in-house effort has 
attempted to identify: (1) Whether any cognitive learning 
has taken place; (2) Long and short term changes in attitude 
of the participants; (3) Changes in the behavior of the 
trained leader, and; (4) Changes in the performance of the 
leader and the organization. 

Preliminary results indicate that with respect to 
the first three criteria the school has achieved some success. 
It was found that four months after training there was still 
some learning going on as the students were generalizing the 
concepts to new situations. [Wehrenberg, 1980, Interview] 
Participants showed significant gains in the other criteria 
when compared to a control group. 

The attitudinal and behavioral criteria were measured 
with questionnaires administered to superiors, peers, and 
subordinates of the trained leader. Individual performance 
was measured with efficiency reports and special reports by 
the superior. These results however suffer from their being 
subjective opinions of one or two people at most. For 
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reasons to be discussed in the next chapter the organizational 
performance criterion has been the most difficult to measure. 
Frequent mission changes were cited as only one reason for 
this difficulty. Future phases of the evaluation process 
call for a detailed examination of this area. The longitudinal 
study continues as the one-year follow-up is about to be made. 

The control group in the above study was (is) comprised 
of individuals who have applied to attend the school but for 
some reason were, or have as of yet, not been accepted. 

Between group comparisons of the control group and individ- 
uals who did not volunteer revealed no significant difference 
which would otherwise invalidate the selection of the control 
group in this manner. 

It is apparent that the Coast Guard is far ahead 
of the other services in evaluating its leadership training. 
While the argument can be advanced that its small size permits 
this type of assessment, it is an inadequate justification for 
failing to follow suit. 

F. SUMMARY 

Based on the preceeding discussion of leadership theories ' 
and training within the military several observations can be 
made concerning the service programs . It was shown that the 
preponderance of current theoretical research recognizes, in 
one form or another, the situational variance in leadership. 
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In the military the most obvious variance in leadership 
roles is the rank of the individual. 

Only two services, the Army and Navy, address this sit- 
uational aspect in their leadership doctrine. This doctrine 
is reflected in a cradle- to-grave concept of instruction, i.e. 
individuals receive a modified version of the base line 
doctrine at each ascension point or formal school. The 
immediate posttraining assignment reinforces these skills. 

The Air Force, while stressing the Mersey and Blanchard 
theory at Squadron Officer School (SOS), fails to fully 
appreciate the complexity of situational leadership in that 
the subject is not specifically addressed in post-SOS 
schooling. The approach taken by the Marine Corps is even 
less cognizant of the power of viewing leadership situationally 
Relying on the traits and principles which have been used for 
many years, the Corps' concern for tradition insures a safe, 
non- threatening doctrine which will be supported by its leaders 
This philosophy is obviously the reason for subsumming all 
human relations training under the banner of leadership, a 
policy which may preclude changes from being made even 
though research supports such changes . 

There is another situational variable which has yet to be 
addressed by any service, that is the variation found in 
different Military Occupational Specialties (MOS) . Are the 
same skills, in the same proportions, identical for the 



123 



infantryman and the military policeman, the commanding officer 
of a destroyer and the commander of an aviation squadron, or 
even the fighter pilot and the bomber pilot? These distinc- 
tions are made primarily on the intuitive appeal of Hersey 
and Blanchard' s theory that leadership style is dependent 
upon the maturity of the follower. Since individuals (enlisted 
personnel) are assigned jobs based on a number of criteria 
(e.g. scores on aptitude tests, level of civilian schooling, 
etc.) it can be assumed that given the same task the maturity 
of the subordinate will vary along at least one dimension 
used to select his/her specialty. 

The Army, with its decentralized approach, is the closest to 
recognizing this distinction but has not progressed far 
enough in the program implementation phase to determine 
whether this distinction will be emphasized. Mixing its 
officers to insure a homogeneous group as the Air Force has 
done overlooks this aspect. Navy standardization policy 
throughout all schools of the same level minimizes the job 
variation in the mix of competencies. Varying the situation 
of a case study by warfare community does not address this 
issue when the emphasis on each dimension is identical. 

This same decentralized approach of the Army's has created 
a situation which makes program development and implementation 
a very lengthy process. With few major distinctions in its 
program, the Navy has approached full implementation having 
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started two years after the Army on the identification of 
leadership competencies. While a longer development phase 
may allow for more input from various sources and hence a 
wider feeling of ownership, the ultimate determination of 
program effectiveness is based on an evaluation after imple- 
mentation. With the proper type of assessment, modifications 
to the initial design are possible to improve the program. 

A program brought on line earlier will obviously reach more 
people and stand a greater chance of making a positive impact 
on the current problems. The Air Force and Marine Corps 
share this benefit of centralization with the Navy.. 

Another benefit the Air Force and Coast Guard have is the 
format of their schools. With individuals returning to their 
command after the course the curriculum developers are able 
to conduct the best evaluation of any military leadership 
program, with the Coast Guard exploiting this advantage to 
its full potential. The same approach is possible in the 
Army's Leadership and Management Development Course and in 
the Navy for the limited number of personnel who attend LMET 
courses on temporary duty from their command but neither 
service has seized this opportxmity . 

With the exception of the Coast Guard, all services are 
subject to criticism for failing to address the necessity of 
conducting comprehensive evaluations. Whether this oversight 
is a result of not recognizing its importance or, as in the 
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case of the Navy, not addressing the need in the initial 
planning the result is the same. Programs which, if called 
upon to justify their existence, would be hard pressed to 
produce conclusive evidence. Although the Army has included 
evaluation as a major objective in its Leader Development 
Plan, it has still been remis in not thoroughly studying the 
effects of L&MDC, As a consequence of this gap in the 
leadership programs, a comparison between services is limited to 
subj ective observations . 

Despite the many differences in the programs there are 
some similarities between the services. First, each service 
has recognized the need to approach leadership training with 
a multitude of instructional techniques, e.g. group discussion, 
role playing, group problem solving, as well as the traditional 
lecture. This signals a positive step in curriculum develop- 
ment in a non-technical area. The Coast Guard's efforts and 
experiences with behavior modeling should advance this position 
even farther. 

Second, and most important, is the recognition of the 
benefit of feedback to the participant on his/her style of 
leadership. Making the most gain in this area is the Air 
Force with their two-week assessment phase. Unfortunately 
the specifics of this assessment methodology suffer from the 
same format previously ascribed to as being beneficial. 

Mixing personnel from various military occupational specialties 



126 



the situations relied upon to point out leadership behavior 
is not reflective of the individual's actual job, e.g. sports. 
One of the principle features in assessment centers is the 
ability to recreate as closely as possible those conditions the 
individual will find on the job, this is far from the sit- 
uation at SOS and has been criticised by participants for 
just this reason. 

In summary, it has been shown that there exists a wide 
variation in the education and training systems of the 
services. Operating the most decentralized system, only the 
Army has required attendance for all officers through the 
staff officer level (upon implementation of CAS^) . The Army's 
focus in its schooling has always been on the next assignment 
while the other services are more concerned with concepts and 
principles to prepare the officer for the remainder of his/her 
career. Figure 11 depicts the differences between the 
services' resident school programs. 

Within these training systems the approach taken towards 
leadership training is just as diversified. Except for the 
Marine Corps and the Coast Guard the services have identified 
a set of leadership dimensions or competencies which contri- 
bute to effective performance. The Navy, Air Force, and 
Coast Guard have sought to increase individual effectiveness 
with organizational performance benefiting from a second 
order effect. The Army, on the other hand, has recently 
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obtained commitment on a base line doctrine centered 
around organizational leadership skills. Figure 12 presents 
the skills and competencies identified for the Army, Navy, 
and Air Force and the principles of leadership of the Marine 
Corps. Figure 13 contrasts the characteristics of each 
service's leadership training program. 

In the next chapter the problems of evaluation noted in 
this summary are addressed in terms of assessment criteria. 
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FIGURE 13. CHARACTERISTICS OF THE 

LEADERSHIP TRAINING PRO- 
GRAMS IN THE MILITARY 



IV. EVALUATION METHODOLOGY 



A. INTRODUCTION 

In designing a strategy to assess the Army's leadership 
training programs it is essential that the purpose of eval- 
uation be clearly defined. Simply stated, the purpose is to 
provide feedback for decision making about a program. Concom- 
itant with this definition is a series of questions which must 
be addressed in order to develop a comprehensive plan. Some 
of these questions include: 

What information is required? 

What sources of information are available? 

Who is to make the decision? 

Who is to conduct the evaluation? 

What evaluation techniques are available? 

When should the data be gathered? 

It is the purpose of this chapter to answer these and 
related questions as they pertain to the US Army leadership 
training programs. To begin this process a distinction must 
be drawn between the two types of evaluation and a determin- 
ation made as to the appropriateness of each. 

An evaluation may be either summative or formative. 
Summative evaluations provide a macro- level assessment of the 
program, that is, whether the program's objectives were 
achieved or not. In this case the relevant decision would be 
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whether or not to continue the program. A sxommative eval- 
uation is appropriate only after full program implementation 
has been reached. At this point the characteristics of the 
course will have had sufficient time to interact with envir- 
onmental influences to determine if the goals were met. 

Formative evaluation, on the other hand, takes a micro- 
level view of the program's components for the purpose of 
providing the decision maker with information on possible 
modifications in order to improve the program. A far more 
complex process than a summative assessment, a formative 
evaluation looks not only at outputs vis-a-vis inputs, it also 
considers the change process itself, in this case leadership 
training. 

Formative evaluation begins with the program's inception, 
monitors its progress through implementation, and provides a 
resource for continuous feedback throughout the program's 
history. Changes in the environment (e.g. societal norms) 
are detected, their impact on the organization analyzed 
(e.g. greater concern for personal freedom), and what aspects 
of the process are being challenged (e.g. Human Relations). 

Clearly, the most desirable type of evaluation is form- 
ative. Unfortunately, as the complexity of the methodology 
increases so does its cost. The following chapter deals 
with this critical issue, an issue which hampered evaluation 
plans in the other services . The complexity of the evaluation 
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also requires that a systematic approach be pursued to analyze 
the interrelationship of the components. 

The components of the Army organizational leadership 
program include the following: 

1. Initial State of the Organization 

This element is the organizational environment from 
which the leaders come prior to receiving training. It is 
presently described by such quantitative variables as: Absent 
Without Leave (AWOL) rates, nonjudicial punishment rates, 
retention rates, percent of Military Occupational Specialty 
(MOS) qualified, etc. It also includes qualitative measures 
such as motivation, esprite, and bearing. Whether these 
indicies are appropriate for studying the effect of leadership 
training is discussed later. 

2 . Pre- training Leaders 

The individuals selected for the training process. 

3 . Process 

These are the properties of the training system which 
include its method of instruction, validity of the dimensions 
of leadership, applicability of the dimensions to the job, 
variation of the emphasis placed on each dimension, etc. In 
the "Black Box" analogy from systems analysis it is here that 
the results of a formative evaluation are fedback. 

4. Posttraining Leaders 

These are the leaders who have participated in the 
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formal training program; the output of the training process. 

A desired result of the program is to produce a positive 
change in the leaders ' behavior when compared to their pre- 
training measurements . 

5 . Future State of the Organization 

The final determination of a program's success is 
the impact the trained leader has on the organization. This 
organization may be the same as the one in the initial state, 
as would be the case in the Leadership and Management Develop- 
ment Course, or different from the one the leader came from 
as in the case of pipeline training (training enroute to a 
new duty station) . Any change in the environment of the 
organization as a result of the influence of a "trained" 
leader will contribute to a redefinition of the initial state 
of the organization for the next iteration of the training 
cycle. In other words, the output for one cycle changes the 
input for the next. 

The relationship between these components of the training 
system constitute the criteria necessary for a formative 
evaluation (figure 14) . These criteria may take the form of 
feedback (such as that provided by either the training part- 
icipants or an expert's opinion) or the measurement of a 
change in some criteria. This latter category includes ;_the„, 
change ii^_knowl^dge about leadership or the learning which 
took place, the change in the leader's attitude toward 
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FIGURE 14. TRAINING EVALUATION 
SYSTEM 



subordinates, peers, superiors, supervisors, mission, leader- 
ship styles, etc., third, a change in the leader's behavior 
with respect to the attitudinal change and increased know- 
ledge (manifested in the utilization of new leadership skills) , 
and finally, a change in the organization's productivity or 
effectiveness as a result of the leader's new behavior. 

These five criteria and the elements of the system are 
depicted on figure 14 as; (1) Process Evaluation; (2) Learning; 
(3) Change in Attitude; (4) Change in Behavior, and; (5) 

Change in Organizational Performance. 

This classification represents a modification and clari- 
fication of Kirkpatrick's [1959] typology of Reaction, Learning 
and Attitudes, Behavior, and Results. 

In the discussion of evaluation programs within the mili- 
tary, reference was made to internal and external criteria. 

This frequently used classification was defined by H. 0. Mann 
[1957] as: 

Internal criteria are those directly linked to the activities 
of the training program, such as attitude scales and the 
achievement tests designed to measure what the program is 
intended to teach. External criteria, are measures designed 
to assess actual changes in job behavior, such as ratings, 
economic indicators, grievance rates, or turnover within 
a manager ' s unit . 



[Campbell, et al. , 1973, p. 274] 
Based on this definition the first three criteria (Process 
Evaluation, Learning, and Attitude) are all internal, the 
latter two external. This distinction is important in that a 
comprehensive evaluation must include both internal and external 
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measures and a comparison of the two. Ideally, all five 
criteria should be utilized, but in reality the resources for 
collecting all of the appropriate data rarely exist. When 
faced with this constraint the next best alternative con- 
sists of a mixture of internal and external criteria obtained 
through any of a number of possible data collection techniques. 

Several frequently used techniques are; questionnaire, 
paper and pencil tests, job sample or performance tests, 
interviews, simulations (to include assessment centers), visual 
observation (expert opinion), performance ratings, and part- 
icipant reports. Throughout this chapter a matrix is developed 
which indicates which technique (s) is (are) appropriate for 
each criteria. 

Bowers and Franklin's [1975J guidelines for acceptable 
data are applicable in determining this appropriateness. These 
include being: (1) Objective, replicable, free from rater bias; 
(2) Reliable, internally consistent, "rather than stable, as 
an alternative definition of reliable, since its change we're 
evaluating" C^owers and Franklin, 1975, p. 158]; (3) Valid, 
techniques measure what they are supposed to measure; (4) 
Practical, obtainable without difficult, expensive, or time 
consuming procedures, and; (5) Useful, productive of "under- 
standable results within the affectable lifespan of the 
program being evaluated" [Bowers and Franklin, 1975, p. 158]. 

One final word in general about criteria, the data 
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obtained by each must be anticipated and planned for in 
advance of the measurement. This includes: (1) the minimum 
acceptable criterion, and; (2) alternatives or options 
based on the results. The first point requires the deline- 
ation of standards which will indicate the necessity for 
change in the program. The two possible standards are, first, 
an absolute standard which fixes a threshold level below which 
program modification is required, and secondly, a comparative 
standard where the significance in the amount of change or 
difference between groups can be determined and must exceed 
chance probability. 

The advantage of the absolute standard is that the decision 
making process requires minimal analysis techniques (i.e. 
the threshold level is either met of not met) . Its dis- 
advantage is that the particular threshold level selected 
may be difficult to justify (why is 90 percent the minimimn 
acceptable level?). In comparison, the control group used 
with the comparative standard is rarely arbitrary or 
abstract, but the analysis technique used to reach the 
decision requires inferential statistics. 

[Alden, 1978, p. 50] 

The desirability of either standard is discussed, where 
appropriate, for the various techniques. 

The second point refers to the need to establish contin- 
gency plans before the evaluation results are known. Whether 
an absolute or comparative standard is used, this foresight 
will aid in determining appropriate modifications to reach the 
intended goal. If the decision maker indicates that no action 
will be taken regardless of the data, then there is obviously 
no need for an evaluation, except perhaps for cosmetic reasons. 
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Although the Amy's base line doctrine (organizational 
leadership) has not been fully integrated into the fomal 
school system, this chapter approaches the program as 
though it has and that a greater degree of standardization 
than now exists will characterize the program in the future. 

In the following discussion of each of the criteria there 
is a common characteristic; the need for clear and specific 
objectives . 

B . PROCESS EVALUATION 

Process evaluation is an assessment of what occurs in the 
classroom and is only tangentially concerned with whether any 
learning has taken place. It is an important criterion 
simply because there must be an atmosphere conducive to 
learning in order for the students to assimilate the infom- 
ation. The atmosphere can broadly be defined as; (1) the 
content of the course, and; (2) the manner of presentation. 
Appropriate sources for these two items are the participants 
themselves and expert opinion based on visual observation. 

In the absence of other criteria Andrews [1966] argues that 
the best indicator of a training program's effectiveness is 
the opinion of the participant. Both the participant and 
expert are currently being used by all of the services as 
their primary means of evaluation. In this section they are 
examined for advantages, disadvantages, and methods of 
application. 
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1. Process Feedback 



There are two forms of feedback from training part- 
icipants, the participant report (or survey) and the inter- 
view, Although the interview potentially can provide the 
most complete data in terms of specific comments and recom- 
mendations, its advantage is importantly dependent upon the 
skills of the interviewer. An experienced interviewer may 
be able to extract more useful information from a fewer 
number of participants than the survey. While this would 
tend to make interviewing a powerful evaluative technique 
there are problems inherent in its use. Principal among these 
problems is the presence of two sources of bias, the part- 
icipant in his/her replies, and the interviewer in the inter- 
pretation of the replies or the manner in which the questions 
are posed. Again the trained interviewer may be able to 
reduce these biases but unfortunately this level of expertise 
is usually not found within the departments responsible for 
gathering such information at the service schools. 

Other disadvantages are the time and dollar resources 
required to conduct interviews. These factors can easily over- 
shadow the advantage of fewer respondents. 

With respect to the Leadership and Management Develop- 
ment Course (L&MDC) conducted at the installation level, the 
survey technique is presently utilized. This despite the Organ- 
izational Effectiveness Staff Officers having received some 
training in interviewing. 
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Critical of the use of the survey as a means of 
gathering participant feedback, Blxomenfeld and Crane [1973] 
comment that "one of the most popular and least meaningful 
criterion measures is participant questionnaires ... this 
approach to evaluation tends to build 'trainer-ego' since it 
produces positive results almost without exception" [Blumen- 
feld and Crane, 1973, p. 43]. To test this assumption they 
conducted a study to document the extent to which perceptions 
of effectiveness of training were based upon quality evidence. 
Their conclusion: "asking managers (or trainees for that matter) 
whether or not training is effective is a meaningless, inappro- 
priate, and ineffective way to proceed" QBliomenfeld and Crane, 
1973, p. 51]. 

Despite this deficiency the written feedback remains 
the most widely used technique. The rationale behind this 
decision is frequently cited as the need for: (1) Data in a . 
standardized format; (2) Unanimity; (3) Ease of administration; 
(4) Data which can be statistically manipulated; and, (5) Qual- 
ity of information in terms of mass distribution of the survey. 
These considerations contribute to the reluctance to use 
interviewing in that it rarely meets these needs. 

Kirkpatrick [1979] remains one of the strongest ) 

supporters of written participant reports for process eval- 
nation. In discussing the advantages of this technique he ^ 
stresses several points which would improve the quality of 
information received. These recommendations are applicable to 
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both the organizational leadership training at the service 
school and the Leadership and Management Development Course, 
First, the report should be distributed prior to the course 
completion. This alerts the participants to the kind of 
information which will be required of them at the end of the 
course. Passing the reports out afterwards may result in 
some lost information. In an extended course (over one week) 
these reports may be collected prior to completion to allow 
for necessary mid-course corrections. The emphasis in the 
report should be on the manner of presentation of the material. 
Depending on the goals of the training, e.g. prepare the 
officer for the next assignment, the participant may not be 
qualified to comment on the relevancy of the course material. 
Any remark along these lines may merely be speculation. The 
L6tMDC, involving personnel who will return to the same duty 
assignment at the end of the course, could be asked in the 
first report about the perceived relevancy of the course. 

An important advantage of the survey technique over 
interviews is the cost savings in subsequent (follow-up) 
participant reports. Several months after training the service 
schools should again query the participants, now graduates, as 
to their perception of the course's effectiveness. In this 
report the emphasis is placed on the relevancy of the mat- 
erial presented to their job. For example, what aspects of 
the training (leadership dimensions) were extraneous, which 
should be emphasized more, and what additions should be made 
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to the course. These follow-up reports will serve as 
primary input for course modifications based on military 
occupational specialty. After several iterations of the 
same training at all schools the training developers will 
be able to adapt the emphasis placed on the leadership 
dimensions so that the infantr 5 mian and the military policeman 
refered to earlier will no longer be receiving identical 
training. 

Tlie second recommendation by Kirkpatrick is that the 
surveys be designed to permit quantification of the responses 
as well as to provide for additional open-ended comments. The 
response categories (scales) should be coded with continuous 
variables (some multiple value variable in a range from high 
to low, e.g. 1 - 5) in order to permit statistical analysis. 

The appropriate agency to conduct the initial and 
follow-up surveys is the agency responsible for conducting 
the training, e.g. the individual service schools for organ- 
izational leadership and the OESO for L&MDC training. Gener- 
alized data concerning the participants' reaction to the 
method of instruction, and later perception of skill rele- 
vancy, should be compiled for research purposes by a desig- 
nated Army-wide organization. At the school level there 
should be a long term trend showing improvement in both content 
and presentation with between group comparisons of student 
reports (providing the reporting procedure and format remains 
constant) . 
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Based on a criteria of complete data the interview 



would tend to be the technique of choice for gathering 
information. However, the serious resource constraints and 
possibility of two sources of bias makes the survey the 
most feasible technique for use by the service schools. 

This concession need not be made in evaluating the L&MDC. 
Follow-up interviews by trained OESO's can be given when 
participants return for an additional session as part of the 
curriculum. Each class is small enough to permit interviewing 
the majority of the attendees, and once the course is completed 
they are all on or near the same installation making a 
second follow-up by the OESO very easy and inexpensive. 

2 . Expert Opinion 

People who are recognized to be subject matter experts 
(either through specialized training or extensive job experience) 
should be called upon to give their perceptions of the leader- 
ship training. Recall that the initial validation of the 
leadership dimensions were made by subject matter experts at 
the Infantry School conferences. 

This technique, like participant feedback, can address 
either the course content (leadership) or the manner of presen- 
tation. While many senior service members would consider them- 
selves qualified to comment on the former, the selection of 
a reviewer should be based on more than rank. It is recog- 
nized than many of the leadership trainers are amply qualified 
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to perform this task, but their involvement in the training 
process would result in a self-evaluation. 

In light of the recent delegation of responsibility 
to the Organizational Effectiveness Center and School for 
the leadership dimensions, and the specialized training 
given to the OESO's in the L&MDC portion of their course, it 
is appropriate that the OESO at the service school be called 
upon to provide input on the process. 

Where possible the reviewer should be capable of 
providing appropriate input (conclusions and recommendations) 
to the Army's leadership research agency (to be determined 
in accordance with the milestones of the Army's Leadership 
Action Plan) as well as to the trainers . In that the 
service schools receive student feedback from which they 
modify the course to fit the situational idiosyncrasies of 
their specialty, this 'outside' expert's role takes on added 
significance. Without this input the course may be modified 
beyond the point of base line doctrine. 

The OESO's are provided expert assistance on course 
content from the Organizational Effectiveness Center and 
School in the form of changes to the manual and their own 
professional publication the OE Communique . 

One of the principle concerns expressed at the Army 
Leadership Action Planning conference was the difficulty 
of structuring leadership training into the Instructional 
Systems Design (ISD) model. Instances in which subject matter 
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experts in training methodology might criticize the instruc- 
tion for failing to meet ISD requirements (see Chapter 3) were 
cited as potential problem areas in future leadership training. 
While this difficulty remains to be resolved at TRADOC the 
leadership instructor must not overlook the potential benefits 
of feedback from experts on methods of instruction, curriculum 
development, etc. Prior coordination with the individual 
selected to evaluate (or inspect) will contribute significantly 
to these benefits. 

Coordination with the post's Directorate of Training 
(or equivalent) may prove helpful for the OESO at an install- 
ation with no service school which can provide this asset. 

The use of expert opinion by the decision maker on 
possible modifications in the program is critical. It is 
not a replacement for, nor is it substitutable by, the 
participant report. Both are important techniques within the 
process evaluation criterion. The following section discusses 
another important internal criterion, learning. 



C . LEARNING 

Learning may be conceived of as a change, due to exper- 
ience, in the students' ways of thinking, feeling, and 
acting. The effectiveness of the learning process may 
be thought of in terms of (1) the magnitude of the 
changes taking place in the individual student or (2) 
the proportion of the students who have changed signif- 
icantly in one or more characteristics relevant to the 
learning process. Thus conceived, education may be 
regarded as a system of learning experiences which 
bring about certain desirable changes in students. 

[Bloom, 1963, p. 386] 
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At the center of the learning process are educational 
objectives which are relatively specific characteristics 
the student should possess after completing the program. 

The learning criterion seeks to determine the participants 
level of knowledge on certain cognitive skills, both before 
and after training relative to these objectives. 

While the curriculum development methodology (figure 8) 
has attempted to specify all leadership competencies in 
terms of cognitive skills there remain only a few which are 
entirely demonstratable on the two learning measurement 
techniques. Through the use of paper and pencil or job 
performance tests the decision maker is provided feedback 
on whether the process (training) is actually teaching those 
skills specified in the objective. 

Regardless of the technique used, there are a number of 
common characteristics. First, a learning test is appropriate 
only where there is a definite correct or incorrect response. 

For example, under the "Directing" subdimension of "Supervision" 
the skill "select appropriate leadership style" is not 
suitable for testing, while "prepare budget" and "write mission/ 
objective statement" are "Management Science" tasks which could 
be tested for compliance with established standards. 

Second, the tests should be designed in accordance with 
accepted doctrine to insure clarity, reliability, validity, 
objectivity, etc. In constructing the tests a principle 
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objective is to insure a particular level of comprehension 
of the material. Therefore, it is appropriate to establish 
an absolute standard for this criteria. Failure to meet the 
standard on the end of course test would indicate either a 
failure of the training process (poor instruction) , a 
discontinuity between the material presented and the objec- 
tives tested (not testing what was taught) , or an unrealistic- 
ally high performance criterion (e.g. 100 percent correct on 
the test needed to pass). 

The third characteristic, the use of pre and posttraining 
tests, clarifies possible alternative explanations. With 
both tests the after training results can be compared to the 
initial test to determine if any change in the participants 
knowledge level can be attributed to the training process. 

A comparison of the mean scores of the tests which proved 
not to be statistically significant would indicate inadequacies 
in the training process (given realistic test standards and 
a failure to meet those standards) . 

The pretest may also reveal that the knowledge level 
prior to training in some skill areas is sufficiently high 
so as not to warrant any additional training in that area. 

As seen in the proposed Military Qualification Standards 
another use of the pretest would be to insure a common base 
of knowledge among the participants before advanced skill 
training is undertaken. 
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While the establishment of a control group (individuals 
not receiving the leadership training) would_increase the 
meaningfulness of the statistical data the appropriateness 
of delaying or exempting personnel from the leadership 
training (a true experimental control group) is a matter 
for the policy makers to decide. If, however, a situation 
is created where the class is subdivided so that two or more 
subdivisions receive the same block of instruction but at 
different times, there is a possibility of conducting an 
institutional cycle "quasi-experiment." This design must 
be considered on a case by case basis to determine what other 
training the first control group is receiving and the amount 
of interchange between the two groups during training, both 
factors which could influence the test results for the group 
receiving the training second. The implications of experi- 
mental design are discussed in greater detail in the section 
on behavioral change. 

Finally, the agency responsible for conducting the test 
should be the service schools for the organizational leader- 
ship and the OESO for L&MDC training. At the service 
school level the ISD subject matter expert should be consulted 
for guidance in test construction. 

Besides the above characteristics, there are some points 
which distinguish the two techniques. In the job performance 
or job sample test the results of the training are measured 
by actual performance (ability) as opposed to strictly cognitive 
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knowledge as with the paper and pencil test. These tests are 
appropriate when a skill is being taught rather than facts 
or principles. For example, if the "Interpersonal Communica- 
tions" skill "speak clearly" is selected for instruction it 
can easily be evaluated by having the participant give an 
oral presentation to the class. 

Disadvantages of job performance evaluation are the amount 
of time it requires, usually having to be administered on an 
individual basis, and/or the need for skilled observers (other 
than the instructor) . A mmierical scoring system must be 
devised to enable statistical analysis of the tests and the 
observers trained in its use to insure validity and relia- 
bility. 

Another form of test, the simulation, is typically more 
general in nature than the task specific job performance test 
and potentially more powerful in providing feedback to both 
the trainee and the decision maker. The assessment center 
approach, as discussed in the final chapter, is a 
variation of the simulation. 

Job performance tests are not easily used to test the 
presence of the learning curve. While paper and pencil tests 
are more flexible in this regard, their result, when adminis- 
tered several months after training may not be a true reflec- 
tion of the educational experience. Having been used 
primarily for evaluating the participants retention of know- 
ledge and facts it can be expected to slope downward from a 
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highpoint immediately after training. The shape of this 
learning curve may be quite different than one obtained from 
a job performance test of the individual's skills. If the 
skills are utilized frequently on the job the curve may be 
expected to show a rise in the skill level since the post- 
training test. 

Less frequent utilization of the skills may produce a 
"sleeper" experience in which a particular skill declines 
(or remains dormant) until some time when a situation arises 
requiring this particular skill. At that time the skill 
level rises to meet the challenge. This phenomenon is 
significant in the measurement of the behavioral criterion. 

Relating this to the military experience, the Navy found 
that its leadership and management' training programs . 
prior to implementation of LMET, concentrated on theories of 
leadership (principles and facts) which proved difficult to 
apply in the Fleet. As a consequence the rate of knowledge 
retention could be expected to sharply decline over time and 
the benefits soon lost. The LMET program is skill based and 
therefore should result in an increase in skill level after 
the participant returns to the Fleet, providing the opportunity 
for reinforcement presents itself. This example serves as 
an important guide to the Army's future program. With 
its intended emphasis on skills of organizational leader- 
ship, vice theory, the technique of choice for determining 
the effectiveness of the training program becomes the job 
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performance test. 

A final comment on the learning criterion; testing, 
besides its use in rank ordering students and evaluating the 
training process, is a powerful tool in determining what is 
learned and how it is learned. It is an integral part of the 
educational process and must be considered as early as the 
development of the program's objectives. 

The introductory remarks to this section refer to the 
change in the students' way of feeling besides the cognitive 
changes due to learning. This emotional, or affective, 
component of learning is measured in the attitudinal criterion. 

D. ATTITUDE 

The measurement of an individual's attitude as an indicator 
of the effectiveness of the training process is more than a 
solicitation of his or her feelings toward the program. This 
was reflected in the participant report . In the domain of 
leadership training the changing of attitudes about the approp- 
riateness of certain leadership styles and a feeling about 
one's subordinates, peers, and superiors may be an objective 
in itself. The review of literature provided several examples 
of this goal. For example, in emphasizing that Theory Y and 
Theory X are attitudes towards people, McGregor held that a 
Theory Y predisposition would allow the leader a greater range 
of behaviors . 

Where the objectives of a training program are expressed 
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in behavioral terms, it is important to recognize that the 
development of a "proper" attitude is an intervening step. 
Simonson [1979] stated "while attitudes are latent and 
not directly observable in themselves, they do act to 
organize, or to provide direction to, actions, and behaviors 
that are observable" [p. 353. 

In measuring attitudes directly, as opposed to infering 
from behaviors, there is an obvious restriction to the 
participant himself /herself as a source of information. The 
techniques available to extract this information include the 
interview and the questionnaire. As with process evaluation, 
the former, when conducted by a skilled interviewer, represents 
the most complete method for gathering data as well as posing 
the greatest problem with respect to the constrained resources 
(people, time, and money). The questionnaire therefore is the 
most likely technique to be used. 

Regardless of the technique used, the first step is to 
identify the construct to be measured. This should follow 
directly from those attitudes which were identified as being 
important during the curriculum development phase and stated 
as a course objective. For example, if there is concern 
about the leadership dimension "Counseling" and the ability 
of the training to impart this concern on the leader, then an 
appropriate construct might be "attitude toward subordinate's 
personal and performance problems." 
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When the constructs have been selected it may be possible 
to use an existing instrument rather than constructing a new 
one. For example, if there was a need to determine an indi- 
viduals concern with htiman relations in the work situation and 
their concern for the task itself, then the Leadership Opinion 
Questionnaire (LOQ) may be used. The LOQ is a paper and pencil 
measure of leadership attitudes which yields scores on the 
two dimensions of leader behavior identified in the Ohio 
State Leadership studies, consideration and initiating struc- 
ture (see Chapter II) . It is completed by the leaders them- 
selves for purposes of self-description. An advantage of 
using existing questionnaires, which may influence the 
selection of constructs, is the availability of norms and 
established measures of validity and reliability. 

If the specific constructs to be studied are not part 
of an existing instrtiment, nor can the necessary items be 
extracted from several sources, it will be necessary to 
develop a new instrument. Such will be the case in the 
likely event that a study of the relationship between the 
eight organizational leadership dimensions and a change in 
the participants' attitude is desired, or that a positive 
attitude towards such dimensions as "Ethics," "Human Relations," 
"Communication," etc. are specified as training objectives. 

The development of an instrxament in this situation is far 
more complex than asking the participant how he/she feels 
about a list of constructs. Issues surrounding the selection 
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of the measurement scale and the test's validity and relia- 
bility are key to constructing a new instrument. Stem [1963] 
raises four questions concerning the scale: (1) Are all items 
relevant to the same measureable continuum; (2) Are the items 
in fact ordered as steps along that continuimi; (3) Is the 
relative distance between the steps constant, and; (4) Are the 
responses actually a function of the attitude the items were 
intended to sample, rather than of some irrelevant process. 

When a test is produced locally, it is critical that val- 
idity and reliability be established through repetitive mea- 
surements and information gathered on the test itself. If an 
existing instrimient is being used, or one has been made from 
several sources, the validity and reliability must be confirmed. 
Both are important methodological issues in assessment and 
require a clear understanding to fully appreciate their sig- 
nificance . 

1. Validity. 

Validity depends on a number of factors and is com- 
prised of several catagories : 

a. Construct Validity. The extent to which a measure 
represents the attitude construct it is supposed to measure. 

One can develop confidence in the value of any construct 
and in the construct validity of a set of measures only 
as the result of a series of experiments in which it is 
found that persons who score 'high' on a test behave 
differently from persons who score 'low' and that this 
difference is in accord with theoretical predictions. 

[Kelly, 1967, p. 48] 
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b. Predictive Validity. How well a measure will 
predict some future behavior. 

c. Concurrent Validity. How the test correlates 
with another instrument which is held to be valid measuring 
the same constructs. 

d. Consensual Validity. Expert agreement that the 
test measures what it purports to measure. 

There have developed over the years several other 
forms of validity which all too often are mistakenly relied 
upon in evaluation research. They are defined here so that 
there is no misunderstanding their inappropriateness as sole 
sources of validity, especially in assessing a training 
program. 

a. Face Validity. A test is believed to measure a 
given construct because it looks like it should. 

b. Validity by Fiat. A test is proclaimed valid 
by its author based on his/her position or reputation. 

c. Experienced Validity. A test is judged valid to 
the extent that it "provides the user with a subjective sense 
of the correctness of the inferences that he makes with it" 
[Kelly, 1967, p. 81]. 

d. Faith Validity. A belief by the user that the 
test yields accurate and correct information therefore it is 
appropriate to make conclusions about other people with it. 



157 



2. Reliability, 



Reliability, or the ability of an instrument to pro- 
duce consistent results can be determined by several methods. 

The "test-retest” method involves the re-administration 
of the instrument to the target group and correlating 
the results. The "split-half” method uses an arbitrary 
division of the instriment into two halves. Results 
from each half are correlated and reported as a relia- 
bility coefficient. "Alternate- form" reliability 
involves the correlation of the results of the same 
attitude construct. Each subject takes each form and 
the resulting correlation is reported as a reliability 
estimate . 



[Simonson, 1979, p. 37] 

The preceeding discussion has called attention to the 
obvious sophistication required to develop a measure of 
attitude test. For this reason the agency which should 
be given responsibility for this phase of the attitude 
criterion is the Army Research Institute (ARI) . Once ARI 
has designed and tested the instrument the administration 
phase is the responsibility of the service school (branch 
schools for the basic officer course. Command and General 
Staff College, or the Army War College) . 

As with the learning criterion, the schools will be 
required to administer a pre- and posttraining test of 
attitude. The question of how long before and after the 
training is difficult to specify. Since attendance would 
necessitate reassignment the change in duty or location might 
influence the participants' overall attitude (motivation) and 
therefore their response. For example, an officer leaving a 
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particularly distasteful assignment would probably experience 
a more positive attitude which would be reflected on the 
instrxament. If the instrxament were mailed to him prior to 
departure the indicator might be more negative. An individual 
leaving a good assignment might react in an opposite manner. 
With respect to the posttraining measurement the attitude 
an individual leaves the school with may be changed or 
reinforced as a consequence of the situation. Even though 
an individuals attitude toward the assignment or the service 
in general is not the same as attitude toward any or all of 
the dimensions of leadership its potential influence can not 
be denied. 

Dressel and Mayhew [1954] , in a study of the gains made 
by college students in attitudes, found a recurring pattern 
of large gains made by students with low pretest scores and 
vice versa. While their explanation of this pattern in terms 
of ceiling effect, regression toward the mean, differential 
motivation, and focus-of-instruction effect as well as the 
statistical procedures available to deal with the phenomena 
in general go beyond the scope of this thesis, it does serve 
to point out the complexity of the issue of when to test. 

In a previous chapter the relevance of differential 
motivation in the military setting was discussed. Clement 
and Zierdt [1973] addressed this issue in the Leadership 
Development Model by noting that soldiers generally enter the 
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service with a positive attitude. This illustrates a point 
of caution with regards to measuring change in attitude. If 
it has been hypothesized that attitude toward some construct 
will change in a positive direction (i.e. viewed more favor- 
ably) as a result of training, and the posttest fails to 
confirm any significant change, has the training process 
failed? Not necessarily. In this situation an absolute 
standard vice the comparative standard must be consulted. If 
the pretest mean was sufficiently high there could be little 
room for change (the ceiling effect) hence the lack of notice- 
able change. At this point the decision maker is confronted 
with the problem of whether or not to continue training 
directed at improving attitude toward that construct when the 
pretest indicates the goal has already been met. 

Despite these problems the most advantageous pretest 
time is immediately upon arrival at the school. The posttest 
should be administered prior to departure from the school 
(to measure attitudinal change which can be more directly 
attributable to the training process and whether this change 
was in the desired direction) as well as some time after 
departure. This longitudinal study will indicate the residual 
or lasting effect of the attitude change once the individual 
has been exposed to the new environment and has had an oppor- 
tunity to assimilate its influence. Changes in this third 
administration of the test may indicate to the decision 
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maker that the original training objectives with respect to 
desired attitudinal change may need modification. This 
decision, however, can only be made when the effects of the 
indivdual's attitude are known. These effects are manifested 
in behavior and performance and are reviewed in the following 
sections . 

E . BEHAVIOR 

What is of interest in this criterion is observed behavior 
in the workplace as opposed to classroom behavior. As the 
first of two external criteria, the best source of data is the 
participant's superiors, peers, and subordinates. Since the 
issue of having one's subordinates report on their superior 
(the training program participant) has been a delicate one 
in the military it must be made absolutely clear to all 
involved that the objective of the measurement is to report, 
in very neutral terms, what has been observed, and is not a 
judgement or evaluation of the leader's effectiveness by 
his/her followers. There are many difficult questions to 
answer in developing this evaluation methodology, a principal 
one being the design of the experiment. 

1 . Organizational Leadership Training 
a. Experimental Design 

The purpose of specifying an experimental design 
is to "make the results less ambiguous and more amenable to 
causal interpretation"[Campbell , et al. , 1973, p, 275]. The 
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authoritative work in this field is Donald T. Campbell 
and Julian C. Stanley's Experimental and Quasi-Experimental 
Designs for Research on Teaching [1963] . It is from this 
publication that much of the design principles for the 
evaluation model recommended for the Army Organizational 
Leadership and Leadership and Management Development Course 
(L&MDC) are derived. 

At the high end of the ambiguity continuum 
(least meaningful results) is the one-group posttraining 
measurement only design. Unfortunately, the ease of 
measurement and minimal cost associated with this design 
has made it very popular. Among its many flaws is its 
failure to provide a base line measurement from which change 
can be determined. Any change that is suspected may be the 
result of any form of influence, before, during, or after 
training . 

A step toward reducing ambiguity is taken with 
the introduction of a pretraining measurement of the part- 
icipants. While a base line is established, it is still 
difficult to determine the cause of any change noted. 
Obviously neither of these one-group designs is suitable for 
formative evaluations. Of the twelve possible sources of 
invalidity Campbell and Stanley [1963] indicate the first 
design fails to control for any of the invalidities and the 
second only for selection and mortality. In 1972 Campbell 
added three more sources of invalidity, the fifteen are as 
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follows : 



1. History. The specific events occuring between the 
before and after measurement in addition to the exper- 
imental variable [the training program] . 

2. Maturation. The ongoing processes within the in- 
dividual, such as growing older or gaining more job 
experience, which are a function of the passage of time 
and which may change the individual in some fashion. 

3. Testing. The effect of the pretest on posttest 
performance. 

4. Instrumentation. The degree to which the criterion 
instruments may measure different attributes of the 
individual at two points in time. An example would be 
using different raters to rate trainee behavior before 
and after training. 

5. Instability. Unreliability of measures, fluctua- 
tions in sampling persons or components, autonomous 
instability of repeated or "equivalent" measures. 

6. Statistical Regression. Changes in criterion 
scores resulting from selecting extreme groups on the 
pretest . 

7. Differential Selection. Using different methods 
to select individuals for the experimental and control 
groups . 

8. Experimental Mortality. The differential loss of 
respondents from the various groups. 

9. Interaction of Differential Selection and Maturation. 
The compounding of the disparity between groups as a 
result of differential selection, with the interval 
changes occurring within the individuals over the course 
of the training period. That is, the experimental 

and control group may have been different to start 
with, and these differences become even greater because 
of interval changes occurring during the experimental 
period. 

10. Interaction of Pretest with the Experimental Variable. 
The possibility that something in the training experience 
reacts with the pretest in such a way that the pretest 

has a greater effect on the trained group than on an 
untrained group . 
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11. Interaction of Differential Selection with the 
Experimental Variable. A different effect for the 
training experience as it is applied to one group versus 
another. Because of differential selection the groups 
are not comparable at the begining on the criterion 
variables, and they may also react differently to the 
training . 

12. Reactive Effects of the Research Situation. The 
phenomenon that exists when the efforts required by the 
research design so change the participant's expecta- 
tions and reactions to the training that results 
cannot be generalized to future applications of the 
training. 

13. Multiple Treatment Interference. The differential 
residual effects of previous training experiences. 

14. Irrelevant Responsiveness of Measures, All 
measures are complex, and all include irrelevant 
components that may produce apparent effects. 

15. Irrelevant Replicability of Treatments. Treat- 
ments are complex and replications of them may fail 
to include those components actually responsible for 
the effects. 



[Campbell, et al. , 1970, p. 277 
& Campbell, D.T. , 1972, p. 191] 

One method of eliminating or controling for some 
of these sources of invalidity requires the introduction of a 
second group, the control group. This group, given the pre- 
and posttraining tests, is not exposed to the training, but 
is exposed to the other environmental factors experienced by 
the group of training participants. When individuals are 
randomly assigned to either group (equal probability of 
assignment) this represents a true experimental design; 
the "Pretest - Posttest Control Group Design." While MacKinney 
[1957] argues that this last procedure is the only one 
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worth the effort [Campbell, et al. , 1970, p. 2763, there is 
still room for error. 

As discussed in the section on Fiedler's 
experiments the Hawthorne effect (heightening of motivation 
as a result of being singled out for training) on the 
training group may account for more of the change recorded 
than the actual training. To control for this effect a 
second control group receiving some training, but not designed 
to teach the same thing, may be used. For example, a group 
of new officers in the experimental group receives the 
organizational leadership instruction, the first control 
group is given the same tests, the second control group is 
given Fiedler’s Leader Match as well as the two tests. If 
there is a difference between the organizational leadership 
group and the first control group, but not with the second 
control group, then the results of the training could most 
likely be attributed to the Hawthorne effect. 

Solomon [1949J added a third control group to 
control for the possible change as a result of the measure- 
ment process itself. This additional group received a post- 
test only while the second control group was given the 
training and the posttest only. 

The detailing of the above four designs has 
served to call attention to several measures which should be 
taken in designing a method to study the effectiveness of a 
leadership training program. Principal among these measures 
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is the use of a control group. At this point in the 
evaluation plan a very critical decision must be made; How 
is the control group to be selected? 

There are two ways in which a control group 
can be designated. The first alternative is the assignment 
to either the control group or the training group as the 
individual "walks through the door." This selection is 
based on some system which makes the probability of going 
to either group equally likely. At the completion of the 
first iteration of training one group has received the train- 
ing which is being studied while the other group (s) have 
not. To illustrate the dilema this alternative poses, the 
Navy point of view will be reviewed. It is the Navy's 
contention that this alternative necessarily denies the 
benefits of LMET attendance to those selected for the control 
group. For this reason the Navy will not use this alternative. 
The Army policy makers must decide whether the benefits derived 
from a scientifically sound experimental design with this first 
alternative outweighs the inconvenience of delaying leadership 
training for one or two hundred leaders during the course of 
the study. If the benefits are deemed insufficient then the 
Army must settle for the second alternative. 

The second alternative method of selecting a 
control group, although it is not as reliable, is to identify 
a naturally occurring control group within the organization. 
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These groups, due to soEe f’jnction of the personnel Eanage- 
nent systen or the training systen and not the action of the 
researcher, did not, or will not, receive the training progran 
being studied. They will, however, be in the saEe environ- 
ment as the training group and available for testing and com- 
parison. 

It is the phenomenon of naturally occurring 



control groups which forms another means of measuring the 
effectiveness of the Army's future organizational leadership 
program. Recognizing that at some point in time the service 
schools will be initiating this program it is proposed that 
the oersonnel attending those service schools orior to 
the anticipated start date be designated as the control 
subjects. The training which they receive in leadership and 
management skills will act as a placebo to minimize the Haw- 
thorne effect. 

There is a requirement that before the new program 
begins a behavioral measurement test be designed snc adminis- 
tered (to whom will be discussed later) prior to the control 
group receiving its training. After this group graduates the 
new program is implemented with the incoming group having had 
the pretest already administered. '.•Tnen this experimental 
group graduates it will enter the iznits and work with peers 
(e.g. platoon leaders) who comprise the control group. After 
a period of assimilation, the subordinates, peers, and superiors 
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of the new platoon leader will be questioned concerning the 
training participant's behavior. This same survey will have 
been administered at the same point in time from date of 
arrival at the unit for the control group. 

The objective is to determine is there is any 
significant difference between the reported observations of 
the control group member's behavior and that behavior report- 
edly displayed by the member of the trained group. In this 
case the groups are the training class of officers at the 
service school, not the platoons or companies they go to. 

A disadvantage of this design is the close mon- 
itoring of subjects, control and training groups, required of 
the research agency. Additionally, this type of experiment 
can only last as long as there are peers already in the units 
who have not had the new training. In the case of the 
platoon leaders the field should be saturated with trained 
lieutenants within two years from the date the service school 
implemented the training. At this point there would cease 
to be a control group and this experiment would terminate. 

The measurement of behaviors could continue but based on one 
of the less desirable forms of validity defined in the pre- 
vious section. This problem does not surface with a true 
experimental control group, the experiment could continue 
indefinitely. 

Having outlined the basic structure of the 
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designs necessitated by the type of control group selected, 
it is now appropriate to detail specifics such as when the 
pretest and posttest should be administered and what tech- 
niques are available. Once these issues are addressed the 
nature of the experimental design is again discussed 
in terms of the recommended course of action, 
b. Measurement Timing. 

The issue which complicated the development of 
an experimental design was the fact that the training program 
under consideration is pipeline training, that is, individuals 
do not return to the same unit after training. Consequently 
the individuals observing the behavior prior to and after 
training are not the same. 

Faced with this problem the only alternative is a 
report by individuals from the unit the participants are coming 
from for the pretraining measurement and a posttraining report 
by people at the unit the trainee was assigned to. This 
resolution is further complicated at the initial entry training 
point; the newly commissioned lieutenants are not coming from 
any unit. 

There are two possible solutions to this: (1) Peer 
measurement before commissioning at the ROTC unit. Military 
Academy, or Officer Candidate School, or; (2) Peer measure- 
ment after commissioning at the officer's basic school. The 
first alternative is selected as the most appropriate for 
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several reasons. First, the peers at the pre- commissioning 
source have had a longer period of time in which to observe 
each other. As noted previously the sleeper effect may 
require a specific situation to trigger a behavior and 
the longer the people know each other the more likely it is 
they would have observed a greater range of behavior. At the 
basic course there is less of an opportunity to make these 
observations prior to the block of instruction on leadership. 

Secondly, a new lieutenant may supress what would 
constitute his/her normal behavior after having been placed in 
a foriegn environment, this of course depending on the indi- 
vidual and/or the commissioning source. 

Finally, the future officer is less anxious 
about completing peer ratings than the newly commissioned 
one. Peer evaluations are a normal requirement at West Point, 
ROTC and OCS and they are less likely to suffer from any 
adverse effects, e.g. distrust, skepticism, etc. 

The use of peer evaluations in leadership studies 

is favorably supported in research. Gordon and Medland [1965] 

found that peer ratings showed higher stability over time 

and over different situations than did the superior's. 

Hollander (1954, 1956, 1957, 1965) and Wherry and Fryer 
(1949) have demonstrated conclusively that peer ratings 
of leadership potential obtained during training in 
military settings have higher reliabilities and are 
more highly related to later ratings of leadership 
competence than other measures or estimates ... obtained 
during training. 

[Campbell et al., 1970, p. 113] 
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Barrett [1966] commented that peer ratings are 



potentially the most accurate judgements of employee job 
behavior. The rationale for these findings is that "a person's 
behavior in the presence of his peers constitutes a more 
valid portrayal of his 'real self than his behavior in the 
presence of his superiors" [Campbell et al., 1970, p. 114]. 

Kelly's [1967, p. 64] comments support the position 
that after the experiment's termination (no more control group 
in the naturally occurring alternative) the peer ratings are 
still desirable, "for many personality variables, the average 
of several peer ratings constitute the best currently avail- 
able measures, and, as such, are frequently used as criterion 
measures against which the scores of newly developed assess- 
ment devices are validated." This also is significant for 
the development of assessment centers in the Army. 

Unfortunately the availability of peer ratings is 
very limited for the posttraining measurement. These data 
must be gathered at some point sufficiently long after the 
leader's arrival to permit assimilation of the environment and 
reaction with the learned skills. It is gathered, as previously 
mentioned, from peer, subordinate, and superior. Obtaining 
information from several levels of the chain of command on an 
individual would, when averaged out, tend to eliminate the 
common biases of observers. 

These biases are: (1) The leniency error, a 
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tendency to rate people favorably; (2) Closure error, using 
knowledge about one behavior to report on one upon which the 
observer has no information, this is both favorably and unfav- 
orably; (3) Similarity error, allowing common characteristics 
between observer and observed to favorably influence the 
rating, and; (4) Contrast error, observers who consider 
themselves good at some skills tend to underrate others on 
that same skill. The degree to which these biases are present 
in the measurement is a function of the technique used to 
gather the information and the training of the rater. 

Before considering the techniques it is now pos- 
sible to return to the experimental design and address the 
third major point as illustrated by Solomon's four group 
design, pretest sensitization. When a group has been exposed 
to a set of relevant factors on a pretest it may learn what 
behaviors to look for on the part of its leader. Solomon's 
solution called for a third control group to receive only the 
posttest and the second group to receive only the training and 
posttest, any difference between groups after the posttest 
could be attributed to this pretest sensitization. The 
experimental design which uses two different groups of raters 
obviates the need for such a control group design. With the 
pre- and posttraining reports coming from different groups 
there is no learning afforded the posttraining respondents. 
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c . Measurement Techniques . 

Similar in nature to the attitudinal criterion, 
the best method for obtaining complete data (in depth and 
detailed) is the interview. As a practical matter, however, 
this is the least feasible method. With respondents scattered 
thoughout the nation's campuses (ROTC) , the Military Academy, 
and Army installations world-wide, the cost of travel alone 
would be prohibitive. This problem is only compounded by 
the presence (or possible presence) of both rater and inter- 
viewer biases. 

Consequently the method of choice is the survey 
or questionnaire. While this technique is subject to the 
respondent's biases discussed, Kelly [1967] reports that 
these biases are normally distributed throughout the respon- 
dent sample and therefore extreme scores tend to cancel each 
other out. Another factor which reduces these errors is the 
validity and reliability of the instrument. One possible 
alternative is the use of existing instruments which have 
already been tested for validity and reliability. 

The Leadership Opinion Questionnaire (LOQ) 
discussed as a measure of an individual's attitude on the 
dimensions of consideration and initiating structure, has a 
companion instrxment for behavior. The Leadership Behavior 
Description Questionnaire (LBDQ) is used by superiors, peers, 
and subordinates to describe the leader's behavior along these 
same two lines. Other tests, such as Hersey and Blanchard's 
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LEAD - Others, have not been sufficiently validated to 
allow consideration be given to their use. 

Another area which deserves exploration is the 
use of an existing organizational survey within the Army 
such as the General Organizational Questionnaire (GOQ) . The 
GOQ is an eighty- four item survey which calls for a measure 
of agreement to behavioral and attitudinal statements, e.g. 
"My supervisor emphasizes mission accomplishment," and 
"My supervisor lets me know when I have done my job well." 

The general catagories of questions concern the unit, super- 
visor, work group, co-workers, and personal satisfaction. 

With only eighteen questions relating to leader behavior 
this survey can not be considered appropriate for providing 
the necessary feedback relative to the new training program. 
Modifying this survey is not a viable alternative since it 
would conflict with the norms and data base being developed. 

A second survey. Command Climate, has twenty 
domains and includes more of the dimensions of organizational 
leadership but is yet only an experimental effort by the Army 
Research Institute (ARI) . The use of this instrument, with 
only minor changes, remains a possible alternative. ARI's 
involvement in this field raises an important question; 

Who will be responsible for administering this phase of the 
evaluation process? 
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d. Research Agency. 

Up to this point in the evaluation plan, the 
service schools have been given responsibility, with the 
assistance of ARI, for conducting the internal evaluation. 
There are two principal reasons why this is no longer possible 
for the external evaluations. First, the task of developing 
a survey, selection and testing of the officers prior to 
training, and monitoring their posttraining assignments is 
too extensive for the service school. It is, however, an 
undertaking which ARI is fully capable of handling. The 
Army Research Institue meets each criterion discussed by 
Eoyang [1977] which must be possessed by the capable research 
agency; competence, objectivity, resources, motivation, and 
credibility. 

The second reason is more politically oriented. 

If the service school were to begin gathering data on officers 
it would be viewed more as an attempt to evaluate the officer 
rather than the program. Cooperation would be difficult, at 
best, to obtain. 

ARI's primary responsibility would be to gather, 
analyze, and feedback the data to the appropriate service 
school. The decision maker (at the school) will then compare 
the results of the stated objectives and make program 
modifications as necessary. 
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e. Design Issues. 

In order to outline the elements of the evaluation 
strategy for the organizational leadership training program the 
notations used by Campbell and Stanley and introduced in 
Chapter II are used. The treatment group is that class 
(e.g. Officer Basic Course) which receives the new training, 
the control group is the class graduating from the school 
prior to implementation of the program (the second alternative 
for selecting the control group) . The evaluation plan is 
depicted as: 

Time > 

Treatment Group : 0^^ ’ X O 2 

Control Group : 0^ O 2 

Again, X represents the training received and the O's indicate 
observation reports received from the respondents. 

Observations at time one (0^) are the peer ratings 
gathered at the precommissioning source, while the observations 
at time two (O 2 ) are from the units (after training) . A stat- 
istical analysis would first compare the two initial obser- 
vations to determine if any pretraining differences exist. 

In their discussion of the Nonequivalent Control Group Design 
(the one most like this) Campbell and Stanley Q1963, p. 217] 
state, "The groups constitute naturally assembled collectives 
such as classrooms, as similar as availability permits but 
yet not so similar that one can dispense with the pretest... 
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the more similar the experimental and the control groups are 
in their recruitment, and the more this similarity is confirmed 
by the scores on the pretest, the more effective this control 
becomes . " 

The important test comes in determining whether 
the change from to O2 is significantly larger (and in the 
desired direction) for the treatment group than for the control 
group. If it is, and the likelihood that this is due to chance 
is small enough (a relative judgement by the decision maker) 
then it can be assumed (within a certain probability) that 
the new training program was responsible for the change. 

The experimental design does contain some defic- 
iencies. Foremost is the possible historical source of 
invalidity. Since the time of observations of the two groups 
do not coincide there may be different environmental effects 
on the two groups. To control for this possibility ARI must 
coordinate the survey administration through the local OESO. 

A brief inquiry should determine if the results of the ques- 
tionnaire are likely to be contaminated by this historical 
invalidity. In any case, since the individuals within each 
group are assigned to separate locations after training there 
is little likelihood that a single event (short of war) 
will affect all of one group. Calculation of mean scores 
for all respondents will help moderate the affect of the 
difference in posttest experiences. 
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A second source of invalidity is the interpret- 
ation of questions asked. With different respondents (college 
peers versus unit personnel) on the pre- and posttraining 
surveys there may be two different interpretations of the 
questions. Control for this effect is best achieved by 
ensuring clear, concise wording on the survey so that there 
is little possibility of misunderstanding what behavior is 
being asked about. A pilot trial of the test instrument can 
help clear up ambiguities and possible misinterpretations of 
the questions asked. Here again, calculation of the mean 
scores will reduce any gross misunderstandings not detected 
in the pilot trial. 

Until such time as a standardized program is in 
use at each precommissioning source the possibility of the 
invalidity of multiple treatment interference exists. At 
the officer basic course level this presents a not so easily 
solved problem. The majority of Military Academy graduates 
attend their branch (occupational specialty) school within 
three months after commissioning. By winter of that school 
year the courses are comprised mostly of ROTC with some OCS 
graduates. This most likely will result in a pretraining 
difference in leadership experience between groups. As was 
discussed in the introduction, one constraint on the eval- 
uation process is the requirement to interface with the 
systems already in place. This aspect of the personnel 



178 



management system represents one such constraint which the 
decision maker must consider in analyzing the data. 

The discussion thus far has centered on the second 
alternative method of selecting a control group, the naturally 
occurring group. This is for two reasons: (1) It is more 

difficult to design an experiment around this type of group, 
and; (2) The implications of not training people in leadership 
along with their peers might erroneously lead a decision maker 
to conclude that it is the method of choice. In the last few 
pages the resulting complexity of the design necessary to 
support this decision has clearly demonstrated its disadvan- 
tages . 

If there is doubt about the capability of this 
design to reduce the ambiguity of the results, and delaying 
training for some leaders is acceptable, then the first 
alternative for selecting a control group is appropriate. 

This group, selected at random from the class scheduled to 
receive the new program, would not receive any leadership 
training at all; a true control group. It should be noted 
that precedence does exist within some service schools for 
exemption from training. 

Should such a procedure be used to select the 
control group the design would be as shown: 

Treatment Group : R 0^^ X ©2 

Control Group : R 0^ O 2 
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This design is weak in only two respects: (1) Interaction of 
the pretest with the experimental variable, and; (2) The 
Hawthorne effect. If the two groups above can each be sub- 
divided then the Solomon four group design discussed pre- 
viously will address these weaknesses. Subdividing only the 
control group and administering a placebo would indicate the 
presence of the Hawthorne effect. Use of a program such as 
Fiedler's Leader Match as the placebo would minimize the 
impact on existing resources. The additional information 
concerning pretest sensitization provided by the Solomon 
design would not be necessary with the difference in the 
pre- and posttraining respondents. The simplicity of these 
designs as well as their ability to account for most sources 
of invalidity obviously makes the true control group the 
preferred alternative. 

The detail with which the organizational leader- 
ship program's evaluation strategy was covered will aid in the 
development of a plan for measuring the behavioral changes 
attributable to the Leadership and Management Development 
Course . 

2 . Leadership and Management Development Course 

The procedure necessary to assess behavioral change 
as a result of the L&MDC is considerably easier to present 
than it was for organizational leadership. Recall that for 
this program the participants attend a one-week course and 
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this design is able to control for such factors as testing, 
selection, and maturation by noting any significant changes 
in the pretraining observations. Plotting the measurements 
taken at each observation can contribute substantially to 
understanding what effect the training may have had. Patterns 
which indicate an unbroken trend throughout all reports will 
give less credence to even a statistically significant change 
between the observations taken at times three and four. For 
a detailed discussion of this measurement technique see D.T. 
Campbell's "Reforms as Experiments." 

Campbell and Stanley []1963] note the lack of 
control which this design has for the interaction of testing 
and training. Obviously the respondent will be learning 
more about behavior and what to watch for the more he/ she is 
asked about it prior to the training of their leader, peer, 
or subordinate. Also noted is the lack of control over the 
interaction of selection and training due to the lack of 
random assignment. While modifications to this design can be 
made to improve control in these areas they would also tend 
to introduce new sources of invalidity as the complexity 
increases . 

b. Measurement Timing. 

Timing for this design is critical. With the 
need to conduct several observations before training the 
lead time becomes considerably longer than people may be 
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accustomed to. The interspacing of the observations is also 
very important. If the time is too short people will consider 
the measurement a test of their memory and attempt to replicate 
their previous report. Too long a time increases the possibil- 
ity of the maturation effect. While Campbell and Stanley say 
that the observational series should hold known cycles 
constant (e.g. paydays, weekly inspections, xmit work cycles, 
etc.), observations which are recorded to coincide with one 
of these cycles may suffer from situational peculiarities. 

For example, monthly observations on payday may elicit 
responses more favorable to the leader than at some other 
time. Regardless of the time interval chosen it should be 
constant throughout the experiment, varying time may vary the 
respondents to an extent that instrumentation error is 
introduced. Weighing the above factors, a monthly observational 
cycle would be the most appropriate. 

c. Measurement Techniques. 

At this level of leadership training the previously 
described benefits of both survey and interview data gathering 
are attainable. Beginning the series of observations with a 
survey the researcher can interview respondents the following 
month to clarify inconsistencies, verify information, etc. 

The interview must provide quantifiable data as well as qual- 
itative in order to make statistical comparisons of the obser- 
vations. The same issues which were raised for the organizational 
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leadership program are applicable to L&MDC, e.g. the use of 
existing instruments and respondent and interviewer biases, 
d. Research Agency. 

The principal beneficiaries of a formative eval- 
uation of L&MDC are the OESO's who conduct the training and 
the Organizational Effectiveness Center and School which 
develops the curriculum. The training which these individuals 
receive (or can receive) qualifies them to carry out the steps 
included in the behavioral criterion. The efforts by other 
agencies involved in evaluation, such as ARI, must be made 
available to the OESO's to aid them in carrying out this 
phase of the assessment. 

3. Campbell, Dunnette, Lawler, and Weick fl970] caution 
about the over reliance evaluators place on experimental 
design and the notion of statistical significance. They 
argue that "demonstrating that a difference between the 
before and after measures is statistically significant is 
only a minimal step. The crucial consideration is whether 
or not the training changes managerial behavior enough to 
make a difference to the organization" [p. 2833- This 
notion of practical significance as opposed to theoretical 
significance is discussed in the final criterion, organ- 
izational performance. 

F. ORGANIZATIONAL PERFORMANCE 

Ascribing changes in organizational performance to leadership 
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or management training is perhaps the most controversial of 
the evaluation criteria. Kirkpatrick [1978, p. 83 states 
"in training areas like leadership and communication there is 
no way that tangible benefits can be directly related to the 
program." In those studies which have used performance results 
Campbell et al. , criticize their methodology by implying that 
behavioral measures are better indicators of training effec- 
tiveness : 

Behaviorally based measures can account for far more 
job complexity and can be related more directly to what 
the manager actually does than the global "organizational" 
measures that have been used in so many training research 
studies done thus far. 

[Campbell, et al., 1970, p. 48l3 
Despite this pessimism there are techniques which can 
contribute to the understanding of the relationship between 
training and results and provide the decision maker with 
information necessary to modify the curriculum. Two tech- 
niques will be reviewed and their interdependence examined, 
the use of performance ratings and the application of cost- 
effectiveness analysis. 

1. Performance Ratings 

The objective of this technique is to identify a set 
of indicators of unit performance which have a high degree of 
correlation with leader behavior. Once identified, comparisons 
between trained leader groups (either program) and untrained 
leader (or old training program) groups along those indicators 
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will tend to show the relative effectiveness of the training 
program. 



For many years, and in many studies, people have 
relied on the traditional indicators of unit effectiveness. 



Motowidlo and Borman ^19783 typify the reliance on these 
measures with their use of the following scales to study morale 
vis-a-vis unit effectiveness; narcotics and drug abuse, total 
serious incidents, absent without leave rates, non- judicial 
punishment rates, congressional inquiries, sick call rates, 
reenlistments, and company inspections. 



Peter Vaill, in answering the question: 



When a group of men using some collection of technologies 
is performing in relation to some predefined goals or 
standards, in a way that may be described as "excellent” 
outstanding" or "high performing," what events may 



or 



be observed in such systems? 



[Vaill, 1978, p. 109] 

has hypothesized 47 indicators of organizational performance. 

Based on this work. Lieutenant Colonel William W. Witt [1979] 

developed a list of 40 descriptors of the High Performing 

System (HPS) which could be used by military commanders to 

measure and improve unit performance. Examples of these 

nontraditional descriptors are: 

There will always be discrepancies between "what the book 
says" and what the HPS actually does. Circumvention of 
the rules tends to be overt and nonapologetic. 

Soldiers in a HPS exhibit reflex behavior to the degree 
that they later cannot account for how or why they acted 
in a particular way. 

[Witt, 1979] 
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Regardless of whether the observer uses the traditional 
indicators or the HPS descriptor^ the concept of using unit 
performance to distinguish between trained and untrained 
leader groups is considerably more complex than transfering 
existing data or checking off HPS behavioral characteristics. 
The primary difficulty in that these measures are dependent 
on a great number of factors, leadership being only one. 

In order for any of them to be indicative of the leader's 
behavior the leader must be able to exert a significant degree 
of influence over their outcome. 

For example, motor pool A takes two days to complete 
repairs to a vehicle, motor pool B requires only one-half 
of a day to successfully accomplish the same task. Does the 
performance of A indicate that the supervisor is lacking in 
ability and needs training in the organizational leadership 
dimensions of "Supervision," "Planning," and "Decision 
Making?" Not necessarily. The difference may be due to the 
leader, or it may be that the mechanics did not have the 
requisite skills, tools, or parts to do the job. Parallel 
examples can be drawn for those indicies used by Motowidlo 
and Borman. 

In statistics this problem is one of identifying the 
percent of variance in the performance measure which can be 
explained by leadership. While methods of determining this 
variance exist (e.g. canonical discriminate functions), their 
explanation goes beyond the scope of this thesis. Suffice it 
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to say that, if research indicates that only a small proportion 
of the difference between units on a particular measurement 
(e.g. AWOL rate) can be attributed to the leader's behavior, 
then that indicator in not an appropriate criteria for eval- 
uating the effectiveness of a particular training program. 

Even though this procedure may seem thorough it may 
not yield the specificity required by the decision maker to 
modify the leadership training program. The need to target 
a certain dimension of organizational leadership, vice only 
the discovery that a difference between units exists, further 
complicates the effort. 

Once, however, the basic research has been accomplished 
and the best indicators of unit performance have been ident- 
ified (best in that they show a high degree of leader influ- 
ence) the evaluation of training may begin. 

a. Experimental Design. 

Similar to those designs previously discussed, 
there is a need to establish a base line measurement, a 
control group, and controls for the various sources of 
invalidity. There is also a need to redefine the treatment. 
Previously the treatment (X) was the training, now, since the 
unit effectiveness is the criteria, the treatment is the inter- 
action of the trained leader and the unit. In this situation 
the appropriate base line measurement at time one (0^) must 
be taken prior to the new leader's arrival in the case of 
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organizational leadership training at the service school, or 
prior to attendance at L<5d-IDC. The second measurenent is taken 
after the leader has tine to assinilate the new environment 
and/or exercise his/her influence, A control grout vill be 
measured during this same time period. This design is depicted 



as : 



Treatment Group : 
Control Grout : 



A 



If other identical units exist then this basic design may be 
expanded in accordance vith the principles outlined under the 
behavioral criterion. 

The important measurements in this section include 
those betveen grout measurements at time one (0,) to determ 
initial difference exists, at 
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and direction of any change. 
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If certain measures require a specific situation (e.g. field 
training) then obviously observations at time one and two 
must wait for the unit to be placed in that environment. 
Restrictions such as this point out the need to develop a 
comprehensive list of indicators. It is envisioned that 
modifications may be made to the basic design if the units 
do not perform the same tasks at the same time (e.g. platoon 
level training in which the units go to the field at different 
times) . In this situation the researcher must evaluate the 
possibility that other sources of invalidity have been 
introduced . 

c. Research Agency. 

Given the complexity of the tasks outlined it is 
obvious that the only agency with the resources available 
to conduct the basic research is the Army Research Institute 
(ARI) . Once the initial research is completed the Organ- 
izational Effectiveness Staff Officer is the appropriate 
staff agency to conduct the field research phase. The inform- 
ation gathered at this level is channeled through Training and 
Doctrine Command to the service schools. 

With the degree to which leadership influences the 
criteria determined, it is possible to use this information 
in other areas. One such area is the study of the cost- 
effectiveness of the training program. 
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2, Cost-Effectiveness Analysis 

It is appropriate that cost-effectiveness is the 
final evaluative technique, for only after the results of 
each of the other criteria are weighed can this measure be 
considered. While some authors suggest that if training can 
not be shown to contribute to the economic goals of the 
organization it should not be conducted [Odiome, 19703, 
others feel "it is probably not possible to demonstrate the 
worth of training in cost accounting terms" [Campbell, et al, , 
1970, p. 272]. In the military such arguments are purely 
academic. Every program should be shown to be cost-effective. 

The objective of this criterion is to compare the 
costs and benefits of the new program to those of the old 
(service school training) or to any other proposed alternative. 
A task made more difficult by the identification of costs 
and the assignment of dollar amoxints to benefits received, 
it is, none the less a feasible task. There are two possible 
approaches to evaluating alternative programs, cost-benefit 
analysis and cost-effectiveness analysis. Levin defines the 
distinction as: 

A crucial assumption for performing benefit-cost analysis 
of alternatives is that the benefits or outcomes can be 
valued by their market prices or those of similar altern- 
atives. Yet, the objectives of many, if not most social 
programs often have no market counterpart ... In such sit- 
uations the effectiveness of a strategy is expressed 
in terms of its actual physical or psychological out- 
come rather than its monetary value. That is, the 
monetary measures of resource costs are related to the 
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effectiveness of a program in producing a part- 
icular impact. When the effectiveness of programs 
in achieving a particular goal (rather than their 
monetary values) is linked to costs, the approach 
is considered to be cost-effectiveness rather than 
cost-benefit analysis. 



[Levin, 1975, p. 92] 

The Army, recognizing the need to rate the effective- 
ness of training programs, has developed a model to structure 
this need. Cost and Training Effectiveness Analysis (CETA) 
is the model designed to assist the decision maker. Developed 
by the US Army Training and Doctrine Command Systems Analysis 
Activity (USA TRASANA) , CETA is formally defined as: 

A methodology which involves dociomented investigation 
of the comparative effectiveness and costs of altern- 
ative training systems for attaining defined perform- 
ance objectives. 



[Neal, 1978, p. 15] 

The approach outlined in the model enables programs 
to be ranked according to their effectiveness relative to 
their costs. It can not directly be used to ascertain whether 
a particular program is worth it in terms of the dollar 
benefits exceeding costs. The reason for this is that costs 
are expressed in monetary terms while benefits under this 
model are expressed in terms of effectiveness in reaching a 
particular objective. 

The CETA seven phase model is comprised of : 
a. Performance Objective Specification. 

This initial step identifies and defines the 
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performance objectives of the training system. This phase 
is in accord with the evaluation plan's initial step of 
establishing program objectives for each criterion in either 
absolute or comparative standards. 

b. Study Conceptualization and Planning. 

This step addresses specific questions to be 
answered by the model in contributing to the overall program 
objectives. Essential Elements of Analysis (EEA) are derived 
and the various alternatives (present training and organ- 
izational leadership training) are compared in detail. 
Responsibility for the remaining phases of the study are 
identifed . 

c. Training Effectiveness Analysis (TEA). 

This third phase identifies Measures of Training 
Effectiveness (MOTE) for comparing alternatives. In this 
plan the MOTE are derived from the discriminate analysis 
results obtained for the performance rating criteria. For 
resource related EEA, Measures of Training Resource Require- 
ments (MTRR) are also identified. The gathering of data 
proceeds along the previously discussed lines and any 
special methodology needed for the study (computers, stat- 
istics, etc.) are identified. In the final steps of this 
phase the information is analyzed to determine if the MOTE 
and MTRR have been met. 
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d. Resource Analysis. 

For this phase those costs attributable to each 
program are calculated. Thomas E. Mirrabel [1978] presents 
a comprehensive model which has been used by the US Civil 
Service Commission to identify training costs. With only 
minor modifications this systematic approach is applicable 
to this phase of CETA. 

e. Cost and Training Effectiveness Analysis. 

This phase integrates the results of analysis 

conducted in phase three and four. Possible comparisons 
include total cost for obtaining a given level of effective- 
ness and the average cost per unit of effectiveness. The 
former is appropriate when the criteria reviewed indicates 
approximately equal level of effectiveness, i.e. organizational 
leadership has not produced any significant change in either 
attitude, behavior, or unit performance. In this case 
the program with the lowest total cost should be retained. 

The latter comparison requires the combination 
of the criteria according to some formula. This formula can 
weigh the criteria in terms of importance to the overall 
objective. For example, all results (for both treatment and 
control groups) are assigned points relative to some scale, 
these points are then multiplied by a criterion weighting 
factor and sxammed for a total measure of program effective- 
ness. The criterion multiples would range from the highest 
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for unit performance to a lower rating for process evaluation 
(opposite the order presented in this chapter). 



Average cost per unit of effectiveness is obtained 



by dividing the total cost by the total points of measure of 
effectiveness. The method "has the desirable quality of per- 
mitting cost-effectiveness comparisons among divergent pro- 
grams with very different characteristics as long as the 
total cost of programs are available and the outcomes of 
the programs are measured in the same effectiveness units 
[criteria]" [Levin, 1975, p. 108]. 



programs are introduced this procedure can be replicated to 
maintain current cost-effectiveness measures, [figure 15] 



a feasible method of evaluating the possible alternative 
leadership training programs , but is this what is needed? 
Given the difficulty in quantifying the benefits of leader- 
ship training it is a necessary first step. 

To the degree that the effects can also be translated later 
into monetary values, a cost-benefit framework can be 
applied at a second stage. Thus, the use of the cost- 
effectiveness approach does permit one to do a cost- 
benefit analysis as well, whenever the physical or 
psychological outcomes can be converted into monetary 
measures . 



f. CETA Reporting. 

Phase six involves documenting the CETA results 
for the decision maker. 



g. CETA Updating. 

As new alternatives or major changes to existing 



It is obvious that cost-effectiveness analysis is 




[Levin, 1975, p. 93] 
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ANALYSIS“ EDUCATIONAL TECHNOLOOY . Gilbert L. Neol, 
March 1978, p. 18 
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G. SUMMARY 



The purpose of this chapter has been to develop a frame- 
work for evaluating the effectiveness of the Army's two leader- 
ship training programs. Towards this end five criteria 
were identified, defined, and their interface with the 
programs examined. These five criteria can be arranged along 
several characteristic scales as shown in figure 16 . 

In the next chapter the details of these criteria are 
extracted and organized into an action plan format. 
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X. CRITERIA 
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TYPE OF 
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SUBJECTIVE OBJECTIVE 


FIGURE 


16. CHARACTERISTICS OF 
EVALUATION CRITERIA 
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V. LEADERSHIP TRAINING EVALUATION ACTION PLAN 



A. INTRODUCTION 

Having detailed the concepts of the evaluation strategy 
in the previous chapter it remains to structure those ideas 
into an action plan format. The purpose of this blueprint 
is to: (1) Identify specific objectives for each criterion; 

(2) List the action steps necessary to reach those objectives; 

(3) Outline responsibilities, and; (4) Align those action 
steps with other ongoing initiatives, specifically the Leader 
Development Plan and the Review of Education and Training 
for Officers Implementation Plan. It is from these two plans 
that this format is derived. 

Two separate plans are detailed, the organizational leader- 
ship training evaluation plan and the Leadership and Management 
Development Course evaluation plan. For both, the criteria 
for selecting a specific technique, apart from those values 
listed in the introduction, were: 

1. Practicality of Data Collection. 

In cases where the prohibiting cost or time required 
to gather the information reduced the likelihood of using a 
specific technique, the method is not considered appropriate. 

2. Meaningfulness of Data. 

Methods which offer the best information in terms 
of freedom from respondent bias and greatest contribution to 
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the reduction of ambiguity are selected as the method of 
choice. The proposed steps are valid regardless of the manner 
in which the control group is selected. 

Proponency for specific actions is assigned within the 
plan at the major command or Army agency level. Further 
assignment is not practical without an exhaustive study of 
the subordinate element's missions. The only exception to 
this policy is the direct tasking of the Army service 
schools where the action is clearly to be implemented at 
their level. 



B. ORGANIZATIONAL LEADERSHIP TRAINING 
1. Process Evaluation 



Objective: To solicit feedback from the trainees 

and subject matter experts on the relevancy of course content 
and the adequacy of presentation. The steps to achieve this 



objective are: 

a. Identify goals of the program 
with respect to information 

of students and subject matter 
experts towards training. 

b. Identify characteristics of 
the program and the trainers 
e.g. methods of instruction. 

c. Identify and train interviewers. 
Structure questions concerning 
student reaction to training 
after presentation of material. 

d. Design student feedback form. 
Provide for quantitative and 
qualitative data. 



TRADOC 



TRADOC 



TRADOC/ 

Service Schools 



TRADOC 
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e. Coordinate ISD methodology for 
leadership training. Modify 
ISD where necessary. 

f. Designate subject matter expert 
for both leadership content and 
instructional technology. 

g. Decision maker may make changes 
in the course of instruction or 
in trainers based on feedback 
from steps c through e above. 

h. Schools provide generalized 
information to TRADOC for 
service-wide modifications. 

2 . Learning 

Objective: To determine the change in 

a result of training. The steps are: 

a. Identify skills to be taught at 
the schools and specify required 
task or knowledge (facts and 
principles) . Establish compar- 
ative standard required to in- 
dicate that learning has taken 
place . 

b. Design a paper and pencil 
instrument for testing of facts 
and principles taught. Specify 
criteria for passing. 

c. Design job performance test for 
demons tr a table types of tasks. 
Specify criteria for passing. 

d. Validate instruments. 

e. Test student's knowledge or 
skill prior to training. 

f. Administer posttraining test to 
students . 

g. Compare pre- and posttraining 
scores to determine extent and 
direction of change. 



TRADOC 

Service Schools 
Service Schools 

Service Schools 

knowledge as 
TRADOC/ 

Service Schools 

TRADOC 

TRADOC 

TRADOC 

Service Schools 
Service Schools 
Service Schools 
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h. Based on criteria set at step a Service Schools 
and the results of step g, 

decision maker modifies course 
as necessary. 

i. Service school provides general- Service Schools 
ized feedback to TRADOC for 

service-wide modifications. 

3. Attitude 



Objective: To determine if the attitude of the 

training participants has changed as a result of training 
and whether this change is significantly different, and in 
the desired direction, from that change experienced by part- 



icipants in the pre-organizational leadership training pro- 



gram (or control group). The steps are: 



a. Identify constructs (attitudes) TRADOC/ 

which are desired in the leader ARI 
as a result of the training. 

Designate a comparative standard 
which will indicate that a 
change has taken place. Indicate 
the required difference between 
programs which will demonstrate 
the superiority of one program 
over the other (inter & intra 
group comparative standards) . 



b. Identify and train interviewers. ARI/ 

Structure attitude questions for Service Schools 

both qualitative and quantitative 

data. 



c. Design attitudinal questionnaire ARI 
to be administered to participants. 



d. Interview participants prior to Service Schools 

start of classes in accordance 
with format developed in step b. 

Distribute questionnaires to the 
participants at this time. 
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e. Mail questionnaires to partici- Service Schools 

pants six to twelve months after 

graduation. 

f. Compare results of pretraining Service Schools 

interviews and questionnaire 

responses (e.g. mean score per 
construct) to results of post- 
training questionnaires. 

Calculate the difference between 
means and their statistical 
significance . 



g. Compare intra-group results to Service Schools 

other groups (control versus 
treatment groups) . Based on 
criteria in step a, determine if 
the attitude change as a result 
of the new program is in the 
direction and to the extent desired. 



h. The decision maker makes appro- 
priate modifications to the 
program based on information 
gathered in g above. 

i. Service schools forward recom- 
mendation to TRADOC. 

j . TRADOC evaluates recommendations 
and modifies the program service 
wide or authorizes the school to 
make necessary changes. 

4. Behavior 



Service Schools 



Service Schools 



TRADOC 



Objective: To determine the extent of behavioral 

change as perceived by superiors, peers, and subordinates 
and ascertain whether this change was a result of the training 
program. The steps within this objective are: 

a. Identify behaviors desired in TRADOC 

leader as a result of training. 

Establish a comparative standard 
which will indicate that one of 
the programs is superior to the 
others . 
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b. 



ARI 



Develop questionnaire to measure 
the extent to which the partici- 
pant displays the behaviors ident- 
ified in step a. 

c. Validate questionnaire. Conduct 
pilot test, 

d. Solicit responses from peers 
prior to graduation based upon 
stratified random sample from 
ROTC, Military Academy, and 
Officer Candidate Schools, 
Stratification should be based 
on the branch assignments in 
order to compare leadership 
skills by occupational duty. 

e. Monitor assignment of personnel 
selected for study through 
service school training to their 
next duty station. 

f. Administer survey to training 
participant's superiors, peers, 
and subordinates. This should 
be conducted six to twelve 
months after arrival at the 
new duty station. 

g. Compare results of pretraining 
surveys with posttraining surveys. 
Calculate difference between mean 
scores for each leadership 
dimension measured and their 
statistical significance. 

h. Compare results of the group which 
received organizational leader- 
ship to the group which didn't. 
Calculate difference between 
groups on both pre- and post- 
training surveys . 

i. Determine if the difference 
between programs is statistically 
significant and in the desired 
direction. Specify by leader- 
ship dimension. 



ARI 

ARI 

ARI 

ARI 

ARI 

ARI 

ARI 
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j . 


Report each service school's 
results to TRADOC. 


ARI 




k. 


Determine which (if any) 


TRADOC/ 






leadership dimensions need 
to be modified. 


Service 


Schools 


1. 


Implement course modifications. 


Service 


Schools 



5. Organizational Performance 

Objective: To determine if the leadership training 

program is reflected in changes in the operational performance 
of the units to which the newly trained leaders are assigned. 
The steps in this criterion are: 

a. Identify indicators of organiza- TRADOC 
tional performance which may be 
attributable to the behavior of 

its leader. Identify Measures 
of Training Effectiveness, 

Essential Elements of Analysis, 

Measures of Training Resource 
Requirements, and the standard 
for program success in this 
criterion . 

b. Conduct a study to analyze the ARI 

indicators selected in step a 

(e.g. discriminate analysis). 

Determine what percent of the 
variance between units on those 
indicators can be attributable 
to the factor leadership. This 
task must be accomplished for 
each ascension level. 

c. To insure valid measures design ARI 
a rating form for use by OESO's 
utilizing the results of step b. 

d. Prior to arrival of the newly OESO 

trained leader record the organ- 
izational performance of the con- 
trol and treatment groups . 
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e. Record performance measures of OESO 

both groups six to twelve months 
after new leader arrival (prior 
to control group leader departure) . 



f. Compare within and between group ARI 
measures to determine if there is 
a difference between the groups 
and whether it was in the desired 
direction. 



g. Review the results of the study 
to determine if performance 
difference criteria is met and/ 
or what modifications may be 
necessary to the programs. 

h. Determine leadership training 
costs for all programs. 

i. Compute average cost per unit 
of effectiveness. 



TRADOC/ 

Service Schools 



Service Schools 



Service Schools 



j . Determine degree to which ARI/TRADOC 

effectiveness indicators can 
be translated into monetary 
terms . 



k. If feasible, compute cost- ARI/TRADOC 

benefit ratio of new and old 

programs . 

l. Determine if minimtim acceptable TRADOC/ 

ratio has been achieved and/or Service Schools 

actions necessary to improve. 



C. LEADERSHIP AND MANAGEMENT DEVELOPMENT COURSE 

Since this program is conducted at the installation level 
the first step the OESO must take is to secure permission 
from the superior unit commander to establish a control 
(comparison) group within his/her command. As can be seen below, 
many of the steps are identical to those taken in the evaluation 
of the organizational leadership training program. For each 
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leadership training evaluation criterion the objective 
remains the same. 



1 . Process Evaluation 

Same as Organizational Leadership 

2 . Learning 

Same as Organizational Leadership 

3. Attitude 



a. Same as Organizational Leadership 

b. Same as Organizational Leadership 

c. Interview participants and members 
of the control group three to six 
months prior to the start of the 
course . 

d. Interview participants and control 
group three to six months after 
course completion. 

e. Compare results of pre- and post- 
training attitudes for both inter- 
views. Calculate difference 
within treatment group and between 
treatment and control group . 

f. Based on the minimum acceptable 
standards, determine if the 
attitudinal change is in the 
direction and to the extent 
required to support the program. 

g. Determine if results of step f 
warrant modification of the course. 
If so, forward recommendations 

to OECS . 

h. Evaluate recommendations and 
modify program as necessary. 



OESO 

OESO 

OESO 

OESO 

OESO 

Organizational 
Effectiveness 
Center & 

School 
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4. Behavior 



a. Same as Organizational Leadership 

b. Same as Organizational Leadership 

c. Develop strategy to interview OEC&S 

superiors, peers, and subordinates 
along measures identified in 

step a. 

d. Same as Organizational Leadership 

e. Select random sample of respondents OESO 
for both control and treatment 

group. Begining three to six 
months prior to the course, conduct 
a series of three interviews and/ 
or surveys of respondents with 
the instrument/ strategy developed 
in steps b and c. Allow a minimum 
of one month between each obser- 
vation. 

f. Between the third and sixth month OESO 
after course completion conduct 

a series of interviews and/or 
surveys of the respondents in step 
e above. Allow a minimum of one 
month between observations . 

g. Compare pretraining and posttrain- OESO 
ing measures. Plot the results 

and examine for trends throughout 
the study. Determine probable 
causes of differences. 

h. Same as Organizational Leadership OESO 
step g. 

i. Same as Organizational Leadership OEC&S 
step h. 

5 . Organizational Performance 

All steps remain the same as those for Organizational 
Leadership, except that the control group leader and the previous 
leader of the treatment group will not have had L&MDC training. 
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D . SUMMARY 



Not addressed thus far has been the subject of milestones 
for the implementation of these steps. The Leader Develop- 
ment Plan calls for the integration and implementation of 
the Leadership Conference ' 79 tasks derived from Monograph 
8 of the Leadership Monograph Series into program of instruc- 
tion development by November 1980. Obviously, if this 
target date is realistic, it is mandatory that the initial 
steps for each criterion be undertaken immediately. Failure 
to do so will result in the lost opportunity to establish 
control measures for the organizational leadership program. 

The majority of the latter steps in each criterion are 
ongoing actions . 

Even if the strategy outlined here is begun in time it 
must be recognized that assessment is a d 3 mamic process. 
Changes in the environment as well as results of the exper- 
iments themselves will force modifications in the evaluation 
plan. The distinction between this situation and the one 
described above is that the decision maker has greater control 
over the course of the study and will be able to weigh the 
consequences of his/her actions. 

In the concluding chapter the implications for future 
modifications of the evaluation plan are examined. 
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VI. CONCLUSION 



A. AREAS OF FUTURE STUDY 
1. Assessment Centers 



Throughout the thesis the subject of assessment 
centers has been alluded to. An assessment center, as illus- 
trated by the US Air Force, is simply a process in which an 
individual's behavior in a number of job related situations 
is observed and evaluated. The results of this evaluation 
may be useful to the decision maker in selecting a person for 
a particular job, acceptance into a program such as Army pre- 
commissioning training, or as Douglas Bray [1976] , a developer 
of AT&T's assessment center, suggests: 

Since many of the goals of management training parallel 
the dimensions utilized for evaluation in assessment 
centers, it seems logical that assessment may provide a 
criterion for the effectiveness of training. [Leaders^ 
could be assessed before and after training, or, in a 
more experimental design, they could be assigned randomly 
to training or no training conditions, or to different 
types of training, and assessed some time after training. 

It may be that the assessment center approach can finally 
throw some light on the overall effectiveness of [leadership] 
training and assist in pinpoii^ing its strengths and 
weaknesses . funderscoring added] 

[Bray, 1976, p. 16-14] 

This potential use of assessment centers warrants a 

closer look at the methodology. It is first necessary to 

point out that assessment is not new to the Army. 

In 1973 - 1974 the US Army Infantry School (USAIS) 

Assessment Center (ACTR) assessed students from the 
Infantry Officer Advanced Course (lOAC) , the Infantry 
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Officer Basic Course (lOBC) , and the Advanced NCO Educa- 
tional System (ANCOES) to determine the feasibility of the 
assessment center as techniques for leadership develop- 
ment and leadership prediction. 

[Dyer and Hilligoss, 1979, p. 1] 

The assessment process took three-and-one-half days 

for each participant. During that time paper and pencil 

tests, simulated leadership tasks, and interviews were used 

by the assessors. Unfortunately the results of the study 

did not support the use of an assessment center. 

Field leadership ratings by superiors, peers, and subor- 
dinates were substantially the same at 6 and 18 months; 
that is, the ratings were reliable. The most assessor 
intensive formal ACTR exercises actually did the poorest 
job of predicting field leadership. Self-description 
provided the most 'leadership predictors and required the 
least assessor and assessee time. 

[Dyer and Hilligoss, 1979, p. 2] 
Since this study considerable research has been 
done in the civilian community and favorable results recorded 
by such corporations as Sears, AT&T, IBM, Standard Oil (Ohio), 
etc. The reasons for the increasing interest in this method- 
ology are: (1) Accuracy of the technique; (2) Powerful 
learning experience for leaders/assessors; (3) High accept- 
ance of the results, and; (4) Fairness of the method. This 
increased interest by civilians was noted in the Review of 
Education and Training for Officers (RETO) study which 
recommended the technique be used to select individuals for 
officer training and that it be studied at the Command and 
General Staff College and the Army War College for career 
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development process. Currently the War College is actively 
developing an assessment center for its students. Colonel 
Anthony Nadal, Director of Curriculum Evaluation and Organ- 
izational Effectiveness, Army War College, has argued that 
the War College is the only school currently capable of 
conducting assessments. The ratio of assessors to partici- 
pants at each of the other service schools he maintains is 
too low to warrant its consideration as a tool to aid in the 
development of the leader. [Nadal, 1980, Interview] 

William Byham of Development Dimensions, Inc. lends 
support to this contention: 

The one-to-two [assessor - participant] ratio has been 
foxand to be the most efficient ratio for most assessment 
centers. While assessors can physically observe more than 
two participants in a group exercise, the one-to-two 
ratio produces almost the maximtim amount of paper work 
than an assessor can be expected to accomplish in an 
assessment program. 

[Byham, 1978, p. 96] 

While the assessor - participant ratio at the Officer 
Basic courses and the Command and General Staff College appears 
to rule out assessment at these levels it must be remembered 
what the objective is. When used to evaluate the effective- 
ness of leadership training the number of participants need 
not overwhelm the resources available. Random selection of 
a portion of the trained and control groups is sufficient for 
the information required. The Army should investigate the 
applicability of assessment methodology at each level of lead- 
ership training. 
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This variation of the assessment center is no dif- 
ferent from the model Colonel Nadal is working on at the War 
College. Both consist of three components: (1) The dimensions 
to be assessed; (2) Assessment techniques, and; (3) The 
assessment staff. [Moses, 1978, p. 5-7] 

The first component has already been identified as 
the eight organizational leadership dimension. To specify 
the particular behaviors of each dimension a front end task 
analysis of the officer positions is required. The Organ- 
izational Effectiveness Center and School has already 
completed this in accordance with the RETO Implementation 
Plan. 

Assessment techniques commonly include; in-basket 
exercises, management games, leaderless group discussions, 
interviews, paper and pencil tests, and role playing. Each 
technique is designed to draw forth certain dimensions or 
behaviors. For a discussion of the techniques and the 
dimensions applicable to each see Lois A. Crooks' "The 
Selection and Development of Assessment Center Techniques" 
[1978]. It is the techniques of the Air Force's assessment 
methodology which have been criticized by the participants. 
They are not convinced that the sports programs accurately 
reflect the dimensions of leadership they are evaluated on. 

The final component, the assessment staff, is the 
biggest investment required in this methodology. Already 
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mentioned was the ratio of staff to participants, other 
issues are; selection, rotating or semi- permanent assessors, 
acquaintance of the staff with the participants, and most 
importantly, assessor training. An interesting aspect of this 
component is the increased leadership skills of those people 
selected and trained as assessors. The process is as much 
a training vehicle for the staff as the participants. 

Moses and Byham's Q19783 Applying the Assessment 
Center Method provides an excellent source of articles con- 
cerning the implementation of assessment centers. Of part- 
icular interest is the report of a task force on developing 
assessment center standards. The report (contained as appendix 
G to this thesis) defines assessment centers, outlines the 
support and training necessary to operate one, and discusses 
the ethics involved in the operations . 

2 . Evaluation Design 

As with all djmamic systems the only certainty about 
the Army's proposed leadership training program is that it 
will change. The first step in evaluation is monitoring the 
program to detect unanticipated or undesired changes. When 
these changes occur, as they inevitably will, the plan will 
require modifications to maintain its feasibility. This form 
of research is most appropriately left to the policy maker. 

Another area greatly in need of further research is 
the identification of measures of organizational perfomance 
which can be linked directly to the behavior of the leader. 
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While the steps outlined in the action plan are clear, the 
time and effort necessary is considerable. Success in this 
area will not only benefit the evaluation of leadership 
training effectiveness but provide the unit commanders with 
badly needed guidance. 

B. SUMMARY 

Leadership training in the Army has come a long way from 
the rote memorization of principles and traits of leadership. 
With contributions from such scientific and academic researchers 
as those reviewed, the service now stands ready to meet the 
challenges of the 1980 's and 1990' s. 

Unfortunately the question of whether or not this course 
of action is providing the better trained leaders has been 
neglected. It was shown that this oversight is not unique 
within the Army, both civilian and military programs suffer 
from the same inadequacies in evaluation (Fiedler and the 
Coast Guard notwithstanding) . The consequence of this 
ambivalence towards assessment is potentially made more severe 
in the face of increasingly limited resources without a 
concomitant reduction in the mission. It will never be known 
if the training is the best possible or if this best effort 
is good enough without an aggressive, scientifically proven, 
evaluation of the program. 

The purpose of this thesis has been to raise the question 
of evaluation and to offer a plan whereby the issue of training 
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effectiveness can be studied. As in any proposal there 
remain issues which must be addressed at the highest levels 
of the organization. Specifically, the matter of control 
groups must be resolved. Does the need for everyone to 
receive leadership training at the same point in their 
careers outweigh the benefits provided by a true experimental 
control? It has been the contention here that such a need 
does not overshadow the requirement to establish a true 
control group. 

The specific evaluation plan proposed here has met the ^ 
six conditions which Campbell, et al., [1970^ express as 
being necessary in any attempt to assess training. These 
are : 

1. Utilization of multiple criteria. .. for the purpose 
of more adequately reflecting the multiple contributions 
of p.eaders^ to the organization's goals. 

2. Some attempts to study the criteria themselves, that is 
their interrelationships and their relationships with other 
relevant organizational variables. The relationship between 
internal and external criteria is especially important. 

If the development program affects the internal criteria 
but not the external criteria, then even though learning 
may be taking place, it apparently does not have any 
relevance for the organization's goals. If the converse 
is true, then the training program apparently has an 
effect on job performance, but not for the reasons intended 
by the trainers. 

3. Enough experimental control to enable the causal arrow 
to be pointed at the training program. 

4. Provisions for saying something about the practical and 
theoretical significance of the results. 

5. A thorough, logical analysis of the process and content 
of the training itself. 
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6. Some effort to deal with the "systems" aspects of 
training impact, that is, how the training effects are 
altered by interaction with other organizational subsystems. 

[Campbell, et al. , 1970, p. 284] 
Bunker and Cohen [1978, p. 11] best summarize the 
contention of this thesis: "'going the extra mile' and 
conducting a comprehensive and well controlled investigation 
may be more mandatory than optional, more investment than 
expense, more income than cost." 
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APPENDIX A 



ORGANIZATIONAL LEADERSHIP DIMENSIONS 
of the UNITED STATES ARMY 



1 . COMMUNICATION 

a . Interpersonal Techniques and Focus 

Effective commanders are able to deliver interper- 
sonal feedback, to "read” nonverbal as well as verbal cues, 
and to utilize informal information networks. 

b. Organizational Techniques and Focus 

Call upon the individual to listen carefully in 
order to assimilate as much factual data as possible; and to 
concentrate on interpreting, interpolating, and synthesizing 
information. 

2. HUMAN RELATIONS 

a. Intergroup Relations 

The ability to deal effectively with individuals 
within a work group (keeps subordinates informed, applies 
rewards equitably, shows interest in subordinates welfare, 
and diagnose subordinate motivation) . 

b. Intragroup Relations 

The ability to work with several groups. 

3 . COUNSELING 

a. Personal Counseling 



Assists individual in developing and implementing 
an action plan enabling them to better handle problem areas. 

b . Performance Counseling 



To convey to the individual the nature of his func- 
tioning on the job. 
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4. SUPERVISION 



a. Procedures 



Organizing, directing, inspecting, advising, explaining, 
maintaining, troubleshooting, and motivating. 

b . Techniques 

5. TECHNICAL 

a. Specific Content Area 

b. Procedures, Techniques, Principles 

c. Focus on Motor Skills 

6 . MANAGEMENT SCIENCE 

a. Procedures 

b. Processes 



Controlling, organizing, development, evaluation, 
problem solving, setting objectives. 

7. DECISION MAKING 

a. Climate Variable 



b . Conceptual Ability 

c. Process and Procedures 

8. PLANNING 

a. Procedures 



Establishing policies, allocating resources, budgeting 
programming, scheduling. 

b. Processes 



Dealing with change, conceptualizing, forecasting, 
strategizing, problem finding. 
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9 . ETHICS 



a. Individual Behavior and Values 

b. Professionalism 

c. Organizational Responsibilities 
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APPENDIX B 



OBJECTIVES of the UNITED STATES ARMY'S 
LEADERSHIP and MANAGEMENT DEVELOPMENT COURSE 

1 . MONDAY 

a. Greater ability to learn from experience. 

b. Interest, involvement and investment in course. 

c. Behavior which contributes to group learning. 

d. Willingness to share relevant personal information. 

e. Accurately diagnose and correct communications 
blockages . 

■ f . Give and receive effective feedback. 

g. Explain own behavior in terms of values. 

h. Communicate non-defensively . 

2. TUESDAY 

a. Communicate directly and accurately with others. 

b. Accurately defines the central issue which the group 
is working either overtly or covertly and behaves in a way 
which facilitates resolution of the issues. 

c. Effectively uses active listening to encourage others 
to talk and to better understand one another. 

d. Avoids or deals effectively with win/lose situations. 

e. Behaves as an effective member of a team. 

f. Behaves in ways which will reduce dysfunctional 
conflict in the group. 
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3 . WEDNESDAY 



a. Successfully influences others' behavior in terms of 
both short and long-term goals, while building his relation- 
ship with those others . 

b. Recognizes the need for functional roles in a group 
and demonstrates ability to take several roles as required. 

c. Uses FIRO theory to explain group behavior and pre- 
dict successful leadership behavior. 

4. THURSDAY 

a. Successfully constructs measureable performance 
obj ectives . 

b. Uses "I" messages and active listening effectively in 
performing cotinseling. 

c. Uses active listening to successfully help another 
solve a personal problem. 

d. Uses non-directive leadership behaviors effectively 
in appropriate situations. 

e. Recognizes the value and uses effective confrontation 
with others. 

5 . FRIDAY 

a. Actively and productively uses Outcomes -- Methods -- 
Resources model. 

b. Indicates interest in the application of Organizational 
Effectiveness techniques in his/her work. 

c. Takes responsibility to reach "closure" with the course. 

6. POST-COURSE 

a. Applies learnings from L&MDC successfully on the job. 
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APPENDIX C 



ADVANTAGES and DISADVANTAGES of the UNITED STATES ARMY'S 
LEADERSHIP and MANAGEMENT DEVELOPMENT COURSE 

1 . ADVANTAGES 

a. Soft data are highly favorable. 

b. LScMDC has a dramatic initial effect, 

c. An environment highly conducive to attitudinal and 
behavioral change is fostered. 

d. Only five days are required to conduct L5cMDC. 

e. A wide range of learning catagories are covered (and 
makes use of current. unit problems). 

f. The content of L&MDC is flexible and can be designed 
to meet the needs of the attendees. 

g. L6cMDC can be used with stranger groups or "family" 
groups, but the latter requires more facilitator expertise 
and cognizance of restructuring and redesign considerations. 

h. Provides opportunity for personal as well as pro- 
fessional growth. 

2 . DISADVANTAGES 

a. Hard data are inconclusive. 

b. Facilitators must be well- trained to deal with what- 
ever intrapersonal or interpersonal conflicts occur. 

c. There is little specific skill practice, 

d. Some threat is experienced by participants. The degree 
depends on the individual participant and the experience of 
the facilitator. 

e. LfitMDC requires two facilitators for every twelve to 
twenty attendees. 
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f. Demand on facilitators is high. 

g. Unit personnel cannot easily be trained to conduct 
in-house training. 

h. Five days of uninterrupted training is required to 
present the complete L&MDC and achieve maximum benefits. 
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APPENDIX D 



COMPETENCIES of the UNITED STATES NAVY'S PROGRAM for 
LEADERSHIP and MANAGEMENT EDUCATION and TRAINING 

1. CONCERN FOR EFFICIENCY AND EFFECTIVENESS 
a. Sets Goals and Performance Standards 



Identification of a problem or concern in terms of 
specific goals. This may include the following: 

(1) Mention of specific goal or definition of a 
revised outcome in terms of an action plan. 

(2) Expression of concern for a unit's standards of 
task performance. 

(3) Mention of an effort to reconsider goals in 
order to make them more realistic. 

(4) Setting of deadlines for task accomplishment. 

The person must personally set goals or clearly 
promote on his or her own the goals set by the organization. 

b . Takes Initiative 

Accepting responsibility of one's own volition or 
taking intiative in accomplishing a task. This includes: 

(1) Being the causal agent in a situation. 

(2) Being proactive rather than reactive or passive. 

(3) Initiating new actions, communications, pro- 
posals, meetings, or directives to accomplish a task. 

(4) Exhibiting resourcefulness and persistence 
(defined as taking two or more actions to circtimvent an 
obstacle) rather than giving up or accepting failure. 

The initiative must not be externally motivated 
(e.g. occasioned by a direct order or emergency situations. 
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2. SKILLFUL USE OF INFLUENCE 



a. Influences 



Successful action to influence others. Pursuit of an 
effective strategy. Specific influence methods and skills 
include : 



(1) Persuasive speaking, briefing, or selling ideas 
by putting them in terms of other's self-interest. 

(2) Building political coalitions or potential 
influence networks . 

(3) Influencing by personal example, i.e. molding 
desirable motivation or behavior (personal charisma) . 

(4) Explaining why, sharing information, or communi- 
cating intent of actions to influence others . 

(5) Gaining commitment to organizational goals, 
traditions, and values by appealing to a higher purpose. 

(6) Making others feel strong. 

The influence strategy must be successful, 
b . Develops Subordinates (Coaches) 

Takes action to instruct, coach, help, train, or 
develop co-workers or subordinates to do their jobs better, 
to be more skillful and responsible in getting the job done, 
or to meet the qualification standards. This may be accom- 
plished in any of the following ways: 

(1) Transferring expertise by setting an example. 

(2) Providing the information necessary to get the 
job done. 

(3) Developing subordinates or supporting their 
self- development efforts by making available to them 
training opportunities, expert help, and resources. The 
expertse must be transferred to the subordinate. 

This competency does not include helping subordinates 
with personal problems. 
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c. 



Team Builds 



Acting to promote a spirit of teamwork and cooperation 
within a work group or with another work group or organization; 
or rewarding contribution to a team effort. This may be 
accomplished in any of the following ways; 

(1) Communicating to others the need for cooperation 
or teamwork, 

(2) Producing teamwork in nonroutine situations 
which require cooperation between and among people and 
work groups in order to accomplish an important task. 

(3) Acting to create s 5 mibols of group identity, 
pride, or team effort, 

d. Self-Control 



Control of impulses, suppression of rage, control 
of emotional involvement, remaining calm in potentially 
explosive situations. The emotions most often controlled 
are anger (exploding at people) and affiliation (getting 
too close to one's subordinates). Some examples include: 

(1) Holding back on an impulse to say or do something 
and replacing impulsive behavior with a more appropriate 
response . 

(2) Not showing anger when under attack. 

(3) Making decisions only after identifying and 
weighing all the facts in the situation. 

(4) Controlling the urge to "do it yourself" and 
instead managing others to take personal responsibility 
which has been assigned to them. 

3 . MANAGEMENT CONTROL 

a. Plans and Organizes 

Planning or organizing activities, people, or 
materials in a hierarchial (priority) , temporal (sequential) , 
spatial, chain of command, administrative, or other order 
that proceeds from a problem to a goal state. The following 
conditions may be present: 

(1) Identification of action steps, resources, or 
constraints involved in reaching an objective. 
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(2) Preparation of a schedule of activities. 

(3) Analysis and prioritizing of alternative courses 
of action. 

(4) Anticipation of specific obstacles before the 
plan is executed, 

(5) Organization of people, materials, or activities 
in a new way to accomplish the task. 

b. Optimizes Use of Resources 

Realistically assessing and weighing the assets and 
limitations of people and tasks before organizing a work 
group for the purpose of maximizing task accomplishment. 

This may occur in the following ways : 

(1) Analyzing the capabilities of individuals and 
characteristics or requirements of jobs and matching people 
and jobs to optimize task performance, 

(2) Fully using available human resources to accom- 
plish meaningful tasks (not "make work"). 

(3) Considering trade-offs between task requirements 
and individual's needs to optimize both performance and 
morale . 



c. Delegates 

Uses the chain of command to get subordinates to take 
responsibility by any of the following means: 

(1) Clearly assigning responsibility for task 
accomplishment to others. 

(2) Using or supporting the use of the chain of 
command to get subordinates to share in task management, 

(3) Encouraging others to seek task management 
responsibility by methods other than direct orders to perform 
a task. 

d. Monitors Results 

(1) Monitoring a work process by seeking information 
regarding task progress or by direct, observation. 

(2) Checking up on the results of one's own or others' 

actions . 



228 



(3) Evaluating the outcome of a task against a 
standard of performance. 

e. Rewards 

(1) Providing positive feedback to others for their 
performance on a specific task. 

(2) Officially citing or recognizing others for their 
meritorious accomplishments, and withholding rewards when tasks 
have not been effectively accomplished. 

f . Disciplines 

Disciplining, and holding accountable, a subordinate, 
or giving someone negative feedback on inappropriate appear- 
ance, behavior, or performance. 

4. ADVISING AND COUNSELING 

a. Positive Expectations 

Expressions of belief or trust in people's basic 
worth or ability to perform, as typified by statements 
involving the following: 

(1) A strong conviction that others are fully 
capable of doing good work when given the chance. 

(2) Generalized positive feelings toward other 
people (e.g. "He was a super Seaman" or I've got the best 
group of PO's you ever saw"). 

(3) Believing that subordinates are valuable 
resources . 

b. Realistic Expectations (Negative Expectations) 

Expression of any of the following as a concern for 
the impact of other's limitations: 

(1) Doubts and concerns about others' ability to 

perform. 

(2) Sensitivity and a realistic concern that orders 
will not be followed or carried out effectively by others. 

(3) Open acknowledgement of these negative expecta- 
tions about the shortcomings of individual or group per- 
formance. 
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(4) Openly stating one's displeasure, disappointment, 
and concern about the shortcomings of individual and group 
performance. 

c. Understands 



Accurately identifying and helping others to under- 
stand what created a problem and what is the real agenda, 
and responding appropriately to get the job done. This type 
of empathy or insight is being able to uncover and accurately 
summarize the feelings which underlie a person's initial 
statements and reports. It is "knowing what other people 
feel" (the ability to accurately assess others' motives, 
thoughts, and behavior or patterns), not "feeling what they 
feel" (personally S3mipathizing with people, which may be 
dysfunctional in getting the job done). Examples are: 

(1) Citing the feelings of another person in an 
accepting fashion (e.g., "I knew he was feeling resentful" 
or "He was disgusted"). 

(2) Mention of understanding the motives of 
another or "figuring someone out," with evidence to back up 
the diagnosis. 

5. CONCEPTUAL THINKING 

a , Conceptualizes 

Identification, through an inductive process, of a 
concept (problem, condition, conclusion, or other state) 
which is greater than merely the sum of the specific examples 
from which the concept is derived. The following elements 
must be present: 

(1) Evidence that the person has rigorously searched 
and identified the available facts. 

(2) Evidence that the person has organized the facts 
so as to draw realistic, plausible inferences about the 
problem. 

(3) Use of two or more alternative concepts which 
taken together, make up an overarching concept ("X versus 
not-X," or X versus Y," where X and Y are parallel concepts). 

(4) An implied comparison of (a) what now exists, 
and (b) what ideally should exist. 

(5) Evidence that conclusions and final judgements are 
based on and supported by the evidence of facts. 
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APPENDIX E 



CHARACTERISTICS OF THE MARINE CORPS LEADERSHIP PROGRAM 



The Marine Corps Leadership Program, with heavy emphasis 
on human relations, is a values oriented program designed to 
improve relationships among Marines, between Marines and in- 
dividuals outside the Corps, and to deal with contemporary 
problems in leadership, discipline, and military profession- 
alism. It is conceived as a training tool that is designed 
to cover all aspects of human understanding and moral character 
as opposed to only race relations. The ultimate objective 
is combat effectiveness. 

The Marine Corps Leadership Program has four basic charac - 
teristics : 

a. The program is guided by research and experimentation . 
The educational materials used in the program, the methods 

of instruction, and the implementation procedures are all 
results of extensive research. Scientific studies, surveys 
and experiments, conducted by the Institute for International 
Research, are provided to Headquarters Marine Corps to update 
and enhance the program. 

b. The program is based on objective values , rather than 
being based on "relativistic" or ''self” theories. The Leader- 
ship Program attempts to rejuvenate a concern for some of 

the basic values we hold as Americans; values such as freedom, 
democracy, equality, and the need for order. The dominant 
theme, the dual- life value, is defined operationally as the 
belief that: we have a duty to others and we have a duty to 
ourselves. This values concept holds that all complex human 
problems, e.g. the women's movement, drug abuse, and family 
breakdown, derive from the same source -- a confusion of 
values. Hence, the starting point for a long-term solution to 
any of our human problems must be the identification of common 
values by all of the conflicting sectors of society, especially 
inside the Corps. 

c. The third characteristic is the guided inter-group 
discussion . The learning discussion group serves as a forum 
for the rational exchange of ideas . It is also an opportunity 
for leaders to provide their Marines with factual information 
to dispel the emotional basis for myths and misunderstandings 
about human differences. 
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d. The fourth characteristic is that the program is 
action oriented . The discussion groups are designed to 
motivate Marines to adjust their behavior in positive ways, 
through interpersonal relations with others. To adjust 
behavior, Marines must develop skills which will facilitate 
positive behavioral adjustments. This is the self-develop- 
ment part of the individual action. 

Contemporary problems of concern to Marines are discussed 
within the framework of the dual- life value philosophy. In 
its simplest terms, the dual- life concept holds that all 
persons have two very strong internal drives, not the only 
drives, but certainly those that every person possesses. 

These drives are the desire for the survival of oneself, to 
include the survival of one's close circle of friends and 
family, and secondly, the desire for the unit or society 
in general to survive. This philosophy provides the leadership 
program with a backdrop against which Marines can discuss 
contemporary problems and begin to formulate solutions. 

[USMC, Leadership , 1979, p. 245] 
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APPENDIX F 



UNITED STATES COAST GUARD 
LEADERSHIP and MANAGEMENT DEVELOPMENT PROGRAM 



MISSION: 

To provide the necessary direction, assistance, and 
support to Coast Guard personnel--line and staff-- 
in supervisory positions ensuring that the most 
effective use is made of the resources--human and other-- 
at their disposal. 

PROGRAM GOAL: 

To make the most effective use of our training and 
resource development dollars by: 

1. Ensuring the availability and use of appropriate, 
professional and consistent leadership and management 
principles and skills throughout the Coast Guard; and 

2. providing methodologies that can be used to develop 
leadership and management skills at servicewide, district, 
and unit levels. 

SPECIFIC OBJECTIVES: 

1. To determine the leadership and management related 
functions and training needs of the service. 

2. To provide a development program which helps 
supervidory personnel deal with their most significant 
leadership and management problems. 

3. To provide appropriate methodologies for leadership 
and management development programs . 

4. To provide an institutionalized program which ensures 
that consistent and relevant leadership and management 
development techniques are available and used at all 
levels . 
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5. To create, throughout the hierarcial structure of 
the service, an atmosphere (attitude and culture) which 
fosters the continued growth of effective leadership 
and management skills . 

6. To provide a means to continually assess the 
changes to organizational climate and performance 
levels which results from leadership and management 
development efforts. 

7. To provide assistance, as requested, in the 
diagnosis, implementation, and assessment phases of 
organizational change efforts within the Coast Guard. 



At the time of this writing the above listed objectives 
were being modified. The following goals have been ident- 
ified but not yet sanctioned. 

To determine the functions and training needs of 
Coast Guard supervisors and managers 

To provide guidance for and coordination of the 
training programs by which supervisory and manage- 
ment knowledge and skills are developed. 

To assess the impact of supervisory and management 
training on the knowledge, attitude, behavior, and 
performance within the service. 

To continually assess the organizational climate 
within the Coast Guard. 

To provide expert advisors to be available to assist 
in the planning, implementing, and assessing of the 
impact of organizational changes on Coast Guard 
personnel . 

To assess the need for and make recommendations about 
the process by which managers and supervisors are 
developed. 

To continually develop personnel with the expertise 
to accomplish these other objectives. 
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APPENDIX G 



STANDARDS FOR ETHICAL CONSIDERATIONS 
for ASSESSMENT CENTER OPERATIONS 



1, Rationale for Assessment Center Standards 



The rapid growth in the use of the assessment center 
method in recent years has resulted in a proliferation of 
applications in a variety of organizational, educational, 
and governmental settings. Serious concerns have been raised 
by many interested parties which reflect a need for a set 
of minimal professional standards for users of this tech- 
nique. The standards should: 

Define what is meant by assessment center 

Describe minimal acceptable practices concerning: 

Organizational support for assessment operations 
Assessor training 

Informed consent on the part of the participants 
Use of assessment center data 
Validation of issues 

These standards are not designed to prescribe specific 
practices . Neither do these standards in any way endorse a 
specific assessment center format or specific assessment 
techniques . Rather we have attempted to provide general 
principles which can be adapted to meet existing and future 
applications . The reader should keep in mind the spirit 
by which these standards were written: as an aid to the 
assessment center user, rather than a set of restrictive pro- 
hibitions . 

2 . Assessment Center Defined 

To be considered as an assessment center, the following 
minimal requirements must be met: 

Multiple assessment techniques must be used. At least one 
of these techniques must be a simulation. 

A simulation is an exercise or technique designed to 
elicit behaviors related to dimensions of perfomance on the 
job by requiring the participant to respond behaviorally to 
situational stimuli. The stimuli present in a simulation 
parallel or resemble stimuli in the work situation. Examples 
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of simulations include group exercises, in-basket exercises, 
and fact-finding exercises. 

Multiple assessors must be used. These assessors must 
receive training prior to participating in a center. 

Judgements resulting in an outcome (i.e. recommendation 
for promotion, specific training or development) must be based 
on pooling information from assessors and techniques. 

An overall evaluation must be made by the assessors at 
a separate time from observation of behavior. 

Simulation exercises are used. These exercises are devel- 
oped to tap a variety of predetermined behaviors and have 
been pre-tested prior to use to insure that the techniques 
provide reliable, objective, and relevant behavioral inform- 
ation for the organization in question. 

The dimensions, attributes, characteristics, or qualities 
evaluated by the assessment center are determined by an analysis 
of relevant job behaviors. 

The techniques used in the assessment center are 
designed to provide information which is used in evaluating 
the dimensions, attributes, or qualities previously determined. 

In summary, an assessment center consists of a standard- 
ized evaluation of behavior based on multiple inputs. Multiple 
trained observers and techniques are used. Judgements about 
behavior are made, in part, from specially developed assess- 
ment simulations. 

These judgements are pooled by the assessors at an eval- 
uation meeting during which all relevant assessment data are 
reported and discussed, and the assessors agree on the eval- 
uation of the dimensions and any overall evaluation that is 
made . 

The following kinds of activities do not constitute an 
assessment center: 

Panel interviews or a series of sequential interviews 
as the sole technique. 

Reliance on a specific technique (regardless of 
whether a simulation or not) as the sole basis for evaluation. 

Using only a test battery composed of a number of 
pencil and paper measures, regardless of whether the judgements 
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are made by a statistical or judgemental pooling of scores. 

Single assessor assessment (often refered to as 
individual assessment) - measurement by one individual using 
a variety of techniques such as paper and pencil tests, inter- 
views, personality measures, or simulations. 

The use of several simulations with more than one 
assessor where there is no pooling of data - i.e. each assessor 
prepares a report on performance in an exercise, and the 
individual reports (unintegrated) are used as the final 
product of the center. 

A physical location labeled as a "assessment center" 
which does not conform to the requirements noted above. 

3 . Organizational Support for Assessment Center Operations 

The assessment center should be administered in a pro- 
fessional manner with concern for the treatment of individuals, 
accuracy of results, and overall quality of the operation. 
Assessment centers should be incorporated as part of a total 
system rather than as a process that operates in a vacuum. 
Considerable care and planning should precede the introduction 
of an assessment center. Policy statements concerning assess- 
ment operations should be formally developed and agreed upon 
by the organization. Minimal considerations in developing 
this policy should include: 

The population to be assessed. 

The purpose of assessment. 

The kinds of people who will serve as assessors. 

The type of training they receive and who is to provide 

it . 

The responsibility for administration of the center. 

Specific restrictions concerning who is to see the 
assessment data, and how they are to be used. 

Procedures for collection of data for research and 
program evaluation. 

Feedback procedure to participants and management. 

Expected "life" of assessment center - i.e. the length 
of time assessment center data will be kept in the files and 
used for decision-making purposes. 
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The professional qualifications (including relevant 
training) of the individual or individuals initially respon 
sible for developing the center. 

4, Assessor Training 



Assessors should receive sufficient training to enable them 
to evaluate intelligently the behaviors measured in the 
center. "Sufficient training" will vary from organization 
to organization and is a function of many factors including: 

The length of time an individual serves as an assessor. 

The frequency of individual participation as an assessor. 

The amount of time devoted to assessor training. 

The qualification and expertise of the assessment 
center trainer. 

The assessment experience of other members of the 
assessment staff. 

The use of professionals (i.e. licensed or certified 
psychologists) as assessors. 

The above list is illustrative of the many issues related 
to assessor training. There is more variability in this area 
than in any other section of the standards. 

While we do not wish to establish minimal standards 
concerning the number of hours of assessor training needed, 
it is difficult to imagine assessors functioning effectively 
with only a one- or two-hour orientation prior to serving as 
an assessor. However, whatever the approach to assessor 
training the essential goal is attaining accurate assessor 
judgements. A variety of training approaches can be used, as 
long as it can be demonstrated that accurate assessor judge- 
ments are obtained. The following minimum training is 
required: 

Knowledge of the assessment techniques used. This 
could include, for example, the kinds of behaviors elicited 
by each technique, relevant dimensions to be observed, 
expected, or typical behaviors, examples or samples of actual 
behaviors, etc. 

Knowledge of the assessment dimensions. This could 
include, for example, definitions of dimensions, relationship 
to other dimensions, relationship to job performance, examples 
of effective and ineffective performance, etc. 
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Knowledge of behavior observation and recording 
including the forms used by the center. 

Knowledge of evaluation and rating procedures 
including how data are integrated by the assessment center 
staff. 



Knowledge of assessment policies and practices of the 
organization, including restrictions on how assessment data 
are to be used. 

Knowledge of feedback procedures where appropriate. 

In addition, some measurement is needed indicating that 
the individual being trained has the capability of functioning 
as an assessor. The actual measurement of assessor performance 
may vary and could, for example, include data in terms of 
rating performance, critiques of assessor reports, observation 
as an evaluator, etc. What is important is that assessor 
performance is evaluated to insure that individuals are 
sufficiently trained to fxanction as assessors, prior to their 
actual duties, and that such performance is periodically 
monitored to insure that skills learned in training are 
applied. 

5 . Informed Consent on the Part of the Participants 



Informed consent is a fundamental concern in conducting an 
assessment center program. This means that the participant is 
given sufficient information prior to assessment to evaluate 
intelligently the nature of the program and the consequences 
of attending or not attending a center. \^ile organizations 
have the right to require participation in an assessment 
program as a condition of emplo)mient or advancement, indi- 
viduals should not simply be "sent” to a center with little 
awareness of why they are going. Rather, they should be 
provided with sufficient information to decide whether or 
not they should attend. 

While the actual information provided will vary from 
organization to organization, the following basic information 
should be given to all prospective participants: 

The purpose of the center and the objectives of 
the program. 

How individuals are selected to participate in the 

center. 



General information about the assessors - the composition 
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of the staff and their training. 

General information concerning the assessment 
process itself. This should include a description of the 
techniques and how the results will be used, the kind of 
feedback given. 

Reassessment policy. 

It is recognized that many assessment center programs 
have descriptive names or titles which are often neutral or 
purposefully general. This is an acceptable practice. How- 
ever, it would be inappropriate to suggest to participants 
that the assessment center is for personal development or 
training when the clear intent is for selection or management 
staffing. 

6. Use of Assessment Center Data 



One characteristic of an assessment center is the volume 
of data produced. There are many different forms of assess- 
ment data, ranging for example, from observer notes, to 
reports on performance in the assessment techniques, to 
assessor ratings, and reports prepared for management. The 
preceding is not exhaustive and could also include participant 
and p$er reports and observations, biographical and test 
data, etc. 

The specific purpose of the reports and data obtained by 
the assessment center should be clearly established. This 
will include a statement concerning individuals who will have 
access to assessment data, the kind of information they will 
receive, and the format that will be provided. 

The recipient of assessment data will be given sufficient 
information or training so that the data provided can be 
clearly interpreted. This will include an estimate of the 
relevance of current assessment data for the use in the 
future . 

The individual assessed should be informed of how the 
assessment data are to be used. This will include: 

Who has access to assessment reports. 

Whether participants will normally receive feedback 
concerning assessment performance. If not, provisions must 
be made to provide such information upon specific request. 

How long assessment data will be retained for operational 
use (as opposed to research use) . 
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7. Validation Issues 



A major factor in the widespread acceptance and use 
of assessment centers is directly related to an emphasis 
on sound validation research. Numerous studies have been 
conducted and reported in the professional literature demon- 
strating the validity of the assessment center process in a 
variety of organizational settings. 

The historical record of the validity of this process, 
however, cannot be taken as a guarantee that a given assess- 
ment program will or will not be valid in a given setting. 
Because of this, each user must ascertain the validity of 
the program as applied to one's organization. The technical 
standards and principles for validation are well documented 
and appear in "Principles for the Validation and Use of 
Personnel Selection Procedures" prepared by the Division of 
Industrial and Organizational Psychology, American Psycho- 
logical Association, 1975, and Standards for Educational 
and Psychological Tests and Manuals" prepared by the American 
Psychological Association, 1974. 

In addition to the above standards which include pro- 
visions related to demonstrating fairness and validity, some 
specific guidelines are provided for assessment center pro- 
grams. These include: 

The ability to document the selection of dimensions, 
attitudes, or qualities evaluated in the center. 

The ability to document the relationship of assess- 
ment center techniques to specific dimensions, attributes, 
or qualities evaluated. 

The ability to document the demographic composition 
of the assessment staff as representative of the group of 
individuals assessed. 

8 . Concluding Statement 

It became obvious in developing these standards that the 
standards should serve as guidelines rather than doctrine. 
Rather than create a set of standards that become ends in them- 
selves, the authors attempted to provide a series of general 
principles which can apply to both managers and professionals 
using this technique. These standards should enable the 
assessment center professional to create, implement, and main- 
tain assessment center programs that protect the rights of the 
individuals while meeting organizational needs at the same 
time . 



[Moses and Byham, 1978, p. 303 - 309] 
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FOOTNOTES 



CHAPTER I 

1. Donald L. Kirkpatrick is professor of Management 
Development at the University of Wisconsin - 
Extension, Milwaukee. He is a past president 
of the American Society of Training Developers 
and has authored numerous articles and books on 
training evaluation. 



CHAPTER III 

2. An indicator variable is a variable from the test 

battery which is either: (1) Significantly correlated 

with competency elements mentioned in a critical 
incident interview; or, (2) Correlated with the 
behaviors reflected in that competency element in 
prior research. 
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